|
| 1 | +# WebGPU Backend |
| 2 | + |
| 3 | +`WebGPUDrawingBackend` is the GPU execution backend for ImageSharp.Drawing. It receives one flush worth of prepared drawing commands, decides whether that flush can stay on the staged GPU path, and if so creates and dispatches the staged scene pipeline against a native WebGPU target. |
| 4 | + |
| 5 | +The WebGPU backend and staged scene pipeline are based on ideas and implementation techniques from Vello: |
| 6 | + |
| 7 | +- https://github.com/linebender/vello |
| 8 | + |
| 9 | +This document explains the backend as a newcomer would need to understand it: |
| 10 | + |
| 11 | +- what problem the WebGPU backend is solving |
| 12 | +- what `WebGPUDrawingBackend` actually owns |
| 13 | +- how one flush moves through the backend boundary |
| 14 | +- where fallback, layer composition, and runtime caching fit into the design |
| 15 | + |
| 16 | +## The Main Problem |
| 17 | + |
| 18 | +The canvas hands the backend a prepared composition scene. That is already a big simplification, but it does not mean the GPU can render that scene directly. |
| 19 | + |
| 20 | +The backend still has to decide: |
| 21 | + |
| 22 | +- whether the target and scene are eligible for the GPU path |
| 23 | +- whether the staged scene can be created safely and legally on the current device |
| 24 | +- when the backend should remain on the GPU and when it should fall back |
| 25 | +- how GPU-only concerns such as native targets, flush-scoped resources, and device-scoped caches fit around the staged raster pipeline |
| 26 | + |
| 27 | +That is why the backend layer exists separately from the staged scene pipeline itself. |
| 28 | + |
| 29 | +The raster pipeline solves "how to rasterize this encoded scene on the GPU". |
| 30 | +`WebGPUDrawingBackend` solves "should this flush go there, and how does the system manage that decision cleanly". |
| 31 | + |
| 32 | +## The Core Idea |
| 33 | + |
| 34 | +The WebGPU backend is a policy and orchestration layer around a staged GPU rasterizer. |
| 35 | + |
| 36 | +Its central idea is: |
| 37 | + |
| 38 | +> decide at the backend boundary whether the flush can stay on the GPU, then either run the staged scene pipeline as one flush-scoped unit of work or fall back cleanly |
| 39 | +
|
| 40 | +That means `WebGPUDrawingBackend` is responsible for entry-point orchestration, not for every low-level GPU detail. |
| 41 | + |
| 42 | +## The Most Important Terms |
| 43 | + |
| 44 | +### Backend |
| 45 | + |
| 46 | +`WebGPUDrawingBackend` is the top-level GPU executor. It owns: |
| 47 | + |
| 48 | +- per-flush diagnostic state |
| 49 | +- staged-scene creation attempts |
| 50 | +- the decision to run the staged path or fall back |
| 51 | +- GPU layer composition when that path is eligible |
| 52 | + |
| 53 | +It is the policy boundary of the GPU path. |
| 54 | + |
| 55 | +### Flush Context |
| 56 | + |
| 57 | +`WebGPUFlushContext` is the flush-scoped execution context for one GPU flush. |
| 58 | + |
| 59 | +It owns: |
| 60 | + |
| 61 | +- the runtime lease |
| 62 | +- device and queue access |
| 63 | +- the target texture and target view |
| 64 | +- the command encoder and optional compute pass encoder |
| 65 | +- the native resources created during the flush |
| 66 | + |
| 67 | +The important lifetime rule is simple: |
| 68 | + |
| 69 | +unless something is explicitly cached in `WebGPURuntime.DeviceSharedState`, it is flush-scoped. |
| 70 | + |
| 71 | +### Staged Scene |
| 72 | + |
| 73 | +A staged scene is the GPU-oriented representation of one flush. |
| 74 | + |
| 75 | +It is produced by the encoder and then consumed by the staged raster pipeline. The staged scene itself is explained in the rasterizer document; from the backend point of view, it is the unit of GPU work that either succeeds as a whole or causes fallback. |
| 76 | + |
| 77 | +### Fallback |
| 78 | + |
| 79 | +Fallback means the backend gives the flush to `DefaultDrawingBackend` and then uploads the CPU result into the native target. |
| 80 | + |
| 81 | +Fallback is part of the design, not an error-handling afterthought. The backend is built to decide as early and cleanly as possible when the staged GPU path cannot or should not run. |
| 82 | + |
| 83 | +## The Big Picture Flow |
| 84 | + |
| 85 | +The easiest way to understand the backend is to follow one normal flush from entry to backend decision. |
| 86 | + |
| 87 | +```mermaid |
| 88 | +flowchart TD |
| 89 | + A[DrawingCanvas.Flush] --> B[WebGPUDrawingBackend.FlushCompositions] |
| 90 | + B --> C[Clear diagnostics] |
| 91 | + C --> D[Try create staged scene] |
| 92 | + D -->|Succeeded| E[Run staged scene pipeline] |
| 93 | + D -->|Failed or unsupported| F[CPU fallback] |
| 94 | + E --> G[Flush completes on GPU] |
| 95 | + F --> H[CPU result uploaded to native target] |
| 96 | +``` |
| 97 | + |
| 98 | +The staged scene pipeline itself is described in [`WEBGPU_RASTERIZER.md`](d:/GitHub/SixLabors/ImageSharp.Drawing/src/ImageSharp.Drawing.WebGPU/WEBGPU_RASTERIZER.md). The important backend point is that the whole GPU flush is treated as one coherent attempt. |
| 99 | + |
| 100 | +## What `WebGPUDrawingBackend` Owns |
| 101 | + |
| 102 | +`WebGPUDrawingBackend` is deliberately smaller than the total amount of GPU code around it. It owns orchestration and policy, not the full details of scene encoding or dispatch. |
| 103 | + |
| 104 | +Its responsibilities are: |
| 105 | + |
| 106 | +- clear per-flush diagnostics |
| 107 | +- try to create a staged scene |
| 108 | +- run the staged path if scene creation succeeds |
| 109 | +- fall back cleanly if any stage fails |
| 110 | +- run GPU layer composition when that path is eligible |
| 111 | + |
| 112 | +The expensive staged work is delegated: |
| 113 | + |
| 114 | +- `WebGPUSceneEncoder` owns scene encoding |
| 115 | +- `WebGPUSceneConfig` owns planning data |
| 116 | +- `WebGPUSceneResources` owns flush-scoped GPU resources |
| 117 | +- `WebGPUSceneDispatch` owns the staged compute pipeline |
| 118 | + |
| 119 | +## The Flush Boundary |
| 120 | + |
| 121 | +`FlushCompositions<TPixel>(...)` in `WebGPUDrawingBackend.cs` is the top-level scene flush entry point. |
| 122 | + |
| 123 | +Its job is intentionally narrow: |
| 124 | + |
| 125 | +- clear the previous diagnostics |
| 126 | +- try to build a staged GPU scene for the current flush |
| 127 | +- run the staged pipeline if that creation succeeds |
| 128 | +- fall back if creation, validation, or dispatch cannot complete safely |
| 129 | + |
| 130 | +Keeping those decisions at the boundary is important. The backend is designed to avoid discovering halfway through execution that one part of the scene ran on the GPU and another must suddenly fall back. |
| 131 | + |
| 132 | +## Scene Eligibility |
| 133 | + |
| 134 | +Before the expensive GPU work begins, the backend performs a scene-level eligibility check through the encoder path. |
| 135 | + |
| 136 | +That check answers the first important question: |
| 137 | + |
| 138 | +"is this flush even a candidate for the staged WebGPU path" |
| 139 | + |
| 140 | +This is distinct from later GPU planning checks. At this stage the backend is only trying to rule out unsupported scene features early, before it starts creating resources or recording dispatches. |
| 141 | + |
| 142 | +## Flush Context Creation |
| 143 | + |
| 144 | +If the scene is eligible, the backend creates a `WebGPUFlushContext`. |
| 145 | + |
| 146 | +This is the point where the abstract prepared scene becomes tied to: |
| 147 | + |
| 148 | +- a concrete device |
| 149 | +- a concrete queue |
| 150 | +- a concrete native target |
| 151 | +- a concrete command encoder |
| 152 | + |
| 153 | +The flush context is also the ownership boundary for flush-scoped GPU resources. When the flush ends, those resources leave with the context unless they are part of device-shared runtime state. |
| 154 | + |
| 155 | +## Fallback As Part Of The Architecture |
| 156 | + |
| 157 | +If any stage fails, `FlushCompositionsFallback(...)` runs the scene on `DefaultDrawingBackend` and uploads the CPU result into the native target. |
| 158 | + |
| 159 | +Fallback covers cases such as: |
| 160 | + |
| 161 | +- unsupported scene features |
| 162 | +- unsupported target or format conditions |
| 163 | +- binding-limit failures |
| 164 | +- resource-creation failures |
| 165 | +- dispatch failures |
| 166 | + |
| 167 | +The architectural goal is simple: |
| 168 | + |
| 169 | +decide as much as possible up front, fall back cleanly, and avoid leaving the flush half-executed across two backends. |
| 170 | + |
| 171 | +## Layer Composition |
| 172 | + |
| 173 | +GPU layer composition is a separate path from staged scene rasterization. |
| 174 | + |
| 175 | +That path lives in: |
| 176 | + |
| 177 | +- `WebGPUDrawingBackend.ComposeLayer.cs` |
| 178 | +- `ComposeLayerComputeShader.cs` |
| 179 | + |
| 180 | +Its job is smaller and different. It composites already-rasterized layer pixels rather than running the full vector scene pipeline. |
| 181 | + |
| 182 | +```mermaid |
| 183 | +flowchart TD |
| 184 | + A[Saved layer] --> B[TryComposeLayerGpu] |
| 185 | + B -->|Native destination supports it| C[ComposeLayer compute shader] |
| 186 | + B -->|Unsupported| D[CPU compose path] |
| 187 | +``` |
| 188 | + |
| 189 | +Keeping layer composition separate from staged scene rasterization makes the backend easier to reason about. |
| 190 | + |
| 191 | +## Runtime And Caching |
| 192 | + |
| 193 | +`WebGPURuntime` and `WebGPURuntime.DeviceSharedState` hold the resources that outlive a single flush. |
| 194 | + |
| 195 | +They cache things such as: |
| 196 | + |
| 197 | +- device access |
| 198 | +- compute pipelines |
| 199 | +- composite pipelines |
| 200 | +- a small amount of reusable device-scoped support state |
| 201 | + |
| 202 | +Everything else in the staged scene path is intentionally flush-scoped. |
| 203 | + |
| 204 | +That split keeps flushes isolated while still allowing truly device-scoped state to be reused. |
| 205 | + |
| 206 | +## Relationship To The Rasterizer Doc |
| 207 | + |
| 208 | +This document stops at the backend boundary and the high-level flush flow. |
| 209 | + |
| 210 | +The staged scene pipeline itself is described in [`WEBGPU_RASTERIZER.md`](d:/GitHub/SixLabors/ImageSharp.Drawing/src/ImageSharp.Drawing.WebGPU/WEBGPU_RASTERIZER.md), including: |
| 211 | + |
| 212 | +- scene encoding |
| 213 | +- planning |
| 214 | +- resource creation |
| 215 | +- scheduling passes |
| 216 | +- fine rasterization |
| 217 | +- chunked oversized-scene execution |
| 218 | +- copy and submission |
| 219 | + |
| 220 | +## Reading Guide |
| 221 | + |
| 222 | +If you want to understand the backend first, read the code in this order: |
| 223 | + |
| 224 | +1. `WebGPUDrawingBackend.cs` |
| 225 | +2. `WebGPUFlushContext.cs` |
| 226 | +3. `WebGPURuntime.cs` |
| 227 | +4. `WebGPURuntime.DeviceSharedState.cs` |
| 228 | +5. `WebGPUDrawingBackend.ComposeLayer.cs` |
| 229 | +6. `WEBGPU_RASTERIZER.md` |
| 230 | + |
| 231 | +That order mirrors the newcomer view of the system: |
| 232 | + |
| 233 | +backend policy -> flush context -> runtime lifetime -> layer composition -> staged raster pipeline |
| 234 | + |
| 235 | +## The Mental Model To Keep |
| 236 | + |
| 237 | +The easiest way to keep this backend straight is to remember that it is not the rasterizer itself. It is the orchestration and policy layer around a staged GPU rasterizer. It decides whether a flush can stay on the GPU, runs that staged path as one flush-scoped unit of work, and falls back cleanly when it cannot. |
| 238 | + |
| 239 | +If that model is clear, the major types fall into place: |
| 240 | + |
| 241 | +- `WebGPUDrawingBackend` orchestrates and decides policy |
| 242 | +- `WebGPUFlushContext` owns one flush's execution state |
| 243 | +- `WebGPURuntime` owns longer-lived device state |
| 244 | +- the staged scene pipeline is a separate subsystem described in the rasterizer doc |
0 commit comments