Presentation using
Reveal.js
. JavaScript is required
Composable Compute Shaders for WebGPU
Daniel McNab
COMP3932 Synoptic Project EPA Presentation
Slides:
https://talks.djmcnab.com/lighters/pwc/
# GPU Compute --- Graphics Processing Units (GPUs) are parallel processors Operate on large datasets efficiently notes: Both energy efficiency and time efficiency (i.e. high throughput) --- ## Use cases - Machine Learning - High fidelity Graphics notes: Machine learning: Machine learning pipelines only perform the same operations.<p>High Fidelity Graphics: Nanite main example, also point clouds. I.e. scientific visualisation --- Operate differently to CPUs Many threads execute the same program in parallel, sharing some work between them notes: There may be 256 or more threads in a workgroup <!-- Image from COMP3221 slides? --> --- Because of the unique execution model, programs do not use the usual programming languages These GPU programs are called compute shaders notes: My project aimed to make these easier to develop </p>
# Why WebGPU? <img src="/webgpu_logo_dark.cbadd854.svg" alt="WebGPU logo" style="max-height:20vh"> Adapted from WebGPU logo by <a href="https://www.w3.org/"><abbr title="World Wide Web Consortium">W3C</abbr></a> --- [WebGPU water simulation](https://github.com/piellardj/water-webgpu) by [Jérémie Piellard](https://github.com/piellardj) <!-- Impressive WebGPU picture - Rend3? Vello? --> <video data-src="https://user-images.githubusercontent.com/22922087/228642866-f7bef392-21a3-464e-99ca-78a793789c3b.mp4" data-autoplay type="video/mp4" style="max-height:40vh"></video> --- ## WebGPU GPU access for the Web Approved by all major browser vendors Supports compute shaders notes: Stable support in Chrome since early May, development in progress for other browsers. <p> Previous attempts to bring compute to browsers didn't get agreement, especially from Apple --- Strong potential to be best portable GPU compute platforn notes: Existing options include OpenCL, and CUDA.</p><p> OpenCL is no longer supported by Apple, and can be challenging to install.</p><p> CUDA programs only execute on NVIDIA GPUs, which increases costs.</p><p> Native programs can use implementations too, especially the wgpu library --- ## Advantages Support running workloads on the user's device more efficiently than on CPUs - Reduces costs - Limits data protection risk - May reduce latencies notes: API only recently available, so use cases still being explored --- ## Projects - <a href="https://github.com/webonnx/wonnx">Wonnx</a>, a machine learning (inference) run-time - <a href="https://github.com/linebender/vello">Vello</a>, a high performance 2d graphics renderer </p>
# Why Composability --- Compute pipelines often use *similar* code Composability allows this similar code to be shared between shaders, making them easier to develop WGSL does not have such composability built-in --- Classes of composability: - Modularity: Where shaders reuse same code - Applicability: Allowing the same code to be used in different contexts notes: Applicability is the name I chose to refer to this form of modularity. <p> Implementations include templates (as in C++) and other systems of type parameters. --- ## Modularity To apply prefix sum to larger datasets, a multi-dispatch algorithm is used <img src="https://developer.nvidia.com/sites/all/modules/custom/gpugems/books/GPUGems3/elementLinks/39fig06.jpg" alt="Blocks of Parallel Scan" class="r-stretch"> <small>Image: [NVIDIA GPUGems](https://developer.nvidia.com/gpugems/gpugems3/part-vi-gpu-computing/chapter-39-parallel-prefix-sum-scan-cuda)</small> notes: Each layer of the diagram is a different dispatch, each requiring their own shader </p><p> However, these shaders each reuse the same prefix sum code </p><p> Without modularity, the implementation of the base prefix sum would need to be included directly within each of these shaders, increasing the likelihood of mistakes --- ## Applicability Parallel scan is prefix sum with an arbitrary operation and data type <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Hillis-Steele_Prefix_Sum.svg/1920px-Hillis-Steele_Prefix_Sum.svg.png" alt="Workgroup wide prefix sum" class="r-stretch" style="background-color:#f5f5f5;border-radius:2%;padding:4px"> <small>Image: [Scott Pakin on Wikipedia](https://en.wikipedia.org/wiki/Prefix_sum#/media/File:Hillis-Steele_Prefix_Sum.svg)</small> notes: This can be used for string comparison, stream compaction and sorting </p><p> However, the version shown in the diagram only supports prefix sum --- WGSL Shader modules only consist of a single string Functions can only have a single signature notes: WGSL is the shader language of WebGPU</p><p> This string is generally in a file. Single string - no modularity</p><p> Single signature means no applicability --- Existing solutions based on string manipulation This is fragile, and does not easily support applicability notes: Project explores a potential solution </p>
# Solution --- <img src="https://www.rust-lang.org/static/images/rust-logo-blk.svg" alt="Rust Logo" style="background-color:#f5f5f5;border-radius:50%"> Uses Rust - a modern programming language Built-in support for the required composability Dependencies and Generics notes: Generics allow functions to be called with arguments of different types, so long as they implement all required traits --- ## Research direction Apply these Rust features to WebGPU shaders Without developing a compiler --- ## Goals - Generate Shaders using Rust - Implement reusable GPU compute algorithms - Determine whether presents better developer experience - Support migration to and from this project --- ## Solution Rust library crate Developers of shaders use this crate Host functions create shader functions Host functions use the composability of Rust Uses the `naga` library notes: Naga is the shader representation of --- <img src="/fits_in_naga.1fe06b8e.png" alt="How the project fits in Naga. Shows different frontends going from source shader languages to a naga::module, including `lighters`. Also shows several backends going to output shader languages" class="r-stretch"> --- This code *generates* a shader It is not executed on the device directly ```rust fn my_shader_add( _: &mut BlockContext, a: Value<u32>, b: Value<u32>, ) -> Returned<u32> { (a + b).as_return() } ``` --- Comparable to a *compiler*, although the results are embedded Shader function this creates: ```rust fn my_shader_add(p1: u32, p2: u32) -> u32 { return p1 + p1; } ``` --- Can have statements as well as expressions: ```rust let n = cx.local_variable("n"); let i = cx.local_variable_init("i", 0); cx.store(&n, &n_base); cx.loop_(|cx| { cx.if_(&n.load().le(&cx.const_(1)), |cx| cx.break_(), |_| {}); cx.if_( &(n.load() % 2).eq(&cx.const_(0)), |cx| cx.store(&n, &(n.load() / 2)), |cx| { n.load().saturating_mul( &cx.const_(3), cx, &n, Some(|cx: &mut BlockContext| cx.return_(&n.load()).into()), ); cx.store(&n, &(n.load() + 1)); }, ); cx.store(&i, &(i.load() + 1)); }); cx.return_(&i.load()) ``` --- For simple functions this is not ergonomic This is because the syntax Rust statements only permit the normal behaviours But these behaviours aren't suitable for the project For example, in an if statement, all branches need to be followed so they can be included in the shader My research question however was about composability <!--.element: class="fragment"--> notes: This previous example does not demonstrate the composability benefits --- ## Modularity Can import functionality from different modules ```rust use util::saturating_mul; let c = saturating_mul(a, b); ``` - Built-in to the project - Installed from a third-party library - In a local module notes: The third-party library must also use the project --- <!--.slide: data-transition="fade-out slide-in"--> ## Applicability Generator programs can use Rust's generics system ```rust /// Apply `Op` to each element of `v`, /// returning the total pub fn reduce_vecn<V: Vector, Op: ReduceOp<V::Inner>>( _: &mut BlockContext, v: Value<V>, ) -> Returned<V::Inner>; ``` ```rust cx.call(reduce_vecn<UVec2, Sum>, (vec2(10, 15),)); // 25 cx.call(reduce_vecn<UVec4, Product>, (vec4(1, 2, 3, 4),)); // 24 ``` notes: This is the actual call syntax --- <!--.slide: data-transition="fade-in slide-out"--> ## Applicability Generator programs can use Rust's generics system ```rust /// Apply `Op` to each element of `v`, /// returning the total pub fn reduce_vecn<V: Vector, Op: ReduceOp<V::Inner>>( _: &mut BlockContext, v: Value<V>, ) -> Returned<V::Inner>; ``` ```rust reduce_vecn<UVec2, Sum>(vec2(10, 15)); // 25 reduce_vecn<UVec4, Product>(vec4(1, 2, 3, 4)); // 24 ``` --- For applicability, an implementation of parallel scan was also created --- ## Implementation Details of implementation are more thoroughly outlined in the report Each operation in the source functions appends to the naga representation --- Two test programs were created: 1. Used the normal WGSL, as a baseline 2. Used this project --- Each of these programs had two compute pipelines: 1. Implemented a collatz conjecture calculation `n=3n+1` for the first 2^18 integers This used a single dispatch 2. Used parallel scan to sum the first 2^16 integers This used three dispatches
# Results --- ## Goals - Generate Shaders using Rust - Implement reusable GPU compute algorithms - Determine whether presents better developer experience - Support migration to and from this project --- ## Goals - Generate WebGPU shaders using Rust This goal was broadly achieved However, the syntax was much worse than normal Rust syntax <!-- Additionally, certain design decisions had a negative impact E.g. expressions had logical reference semantics, but were implemented with move semantics --> notes: Therefore, many calls required. --- ## Goals - Generate WebGPU shaders using Rust Composability features however were fully supported Overall, this goal was met --- ## Goals - Implement reusable GPU compute algorithms A generic version of parallel scan was implemented Parallel scan was the only algorithm implemented, as this was a research prototype --- ## Goals - Implement reusable GPU compute algorithms The API of this algorithm was clunky <!-- An idiomatic Rust API would use closures However, the design used to allow calling functions did not permit this --> This goal was partially met, but further infrastructure changes are required to fully realise this --- ## Goals - Determine whether presents better developer experience Overhead of syntax was very significant For shaders which can uses generic algorithms, these sections are significantly more concise, and potentially less error-prone --- ## Goals - Determine whether presents better developer experience This goal was met, as an evaluation was performed The scenarios where it is better are unfortunately narrow notes: paths towards improvement are briefly discussed in future work --- ## Goals - Support migration to and from this project Only used standard WebGPU APIs in prototype Therefore each shader can be converted individually The two pipelines used in the test programs were migrated from WebGPU versions Converting from the project can be partially automated However, the results require significant manual effort --- ## Goals - Support migration to and from this project Overall, this goal was achieved However, the experience of migration was much worse than expected --- ## Other results - Performance - Project avoids needs to parse shaders for native use of WebGPU. Measurably decreased startup latency
# Future work --- Improved Ergonomics
# Questions Slides: <a href="/lighters/pwc/">https://talks.djmcnab.com/lighters/pwc/</a>