[Vello Hybrid]: Clipping (Spatiotemporal Allocation) (#957)

## Context

Hooks up and addresses all the TODO's in Raph's sketch
(#934). See this
[thread](https://xi.zulipchat.com/#narrow/channel/197075-vello/topic/Spatiotemporal.20allocation.20.28hybrid.29/near/513442829)[#vello
> Spatiotemporal allocation (hybrid) @
💬](https://xi.zulipchat.com/#narrow/channel/197075-vello/topic/Spatiotemporal.20allocation.20.28hybrid.29/near/513442829).


## Notes

- Made `GpuResources` non-optional - I thought this was more trouble
than it was worth.
- Adds in new `clear_slots` pipeline to support fine grained clearing of
slots in slot textures (needed for spatiotemporal allocation)
- Regenerates test snapshots
- Enables the clipping test suite for `vello_hybrid`
- There are many performance wins to be had. This PR is pretty big
already, so I think these are worth following up separately.
- Not re-creating buffers for each render pass (re-using the allocations
between calls)
- Using a staging belt (to prevent allocating an extra staging buffer
per `write_buffer`)
   - Perhaps allowing more than 1 column of slots per slot texture.

I've copied the documentation from `schedule.rs` below for information
about how spatiotemporal allocation works:

# Scheduling

- Draw commands are either issued to the final target or slots in a clip
texture.
- Rounds represent a draw in up to 3 render targets (two clip textures
and a final target).
- The clip texture stores slots for many clip depths. Once our clip
textures are full,
we flush rounds (i.e. execute render passes) to free up space. Note that
a slot refers
  to 1 wide tile's worth of pixels in the clip texture.
- The `free` vector contains the indices of the slots that are available
for use in the two clip textures.

## Example

Consider the following scene of drawing a single wide tile with three
overlapping rectangles with
decreasing width clipping regions.

```rs
const WIDTH: f64 = 100.0;
const HEIGHT: f64 = Tile::HEIGHT as f64;
const OFFSET: f64 = WIDTH / 3.0;

let colors = [RED, GREEN, BLUE];

for i in 0..3 {
    let clip_rect = Rect::new((i as f64) * OFFSET, 0.0, 100, HEIGHT);
    ctx.push_clip_layer(&clip_rect.to_path(0.1));
    ctx.set_paint(colors[i]);
    ctx.fill_rect(&Rect::new(0.0, 0.0, WIDTH, HEIGHT));
}
for _ in 0..3 {
    ctx.pop_layer();
}
```

This single wide tile scene should produce the below rendering:
```
┌────────────────────────────┌────────────────────────────┌─────────────────────────────
│      ──              ───   │       /     /       /     /│        ──────────────      │
│  ────            ────      │      /     /       /     / │────────                    │
│──           ─────          │     /     /       /     /  │                            │
│        ───Red              │    /     /Green  /     /   │           Blue             │
│    ────                ──  │   /     /       /     /    │                     ───────│
│ ───                ────    │  /     /       /     /     │       ──────────────       │
│                  ──        │ /     /       /     /      │───────                     │
└────────────────────────────└────────────────────────────└────────────────────────────┘
                                                                                        
```
How the scene is scheduled into rounds and draw calls are shown below:

### Round 0

In this round, we don't have any preserved slots or slots that we need
to sample from. Simply,
draw unclipped primitives.

### Draw to texture 0:

In Slot N - 1 of texture 0, draw the unclipped green rectangle.

Slot N - 1:
```
┌──────────────────────────────────────────────────────────────────────────────────────┐
│       /     /       /     /        /     /       /     /       /     /       /     / │
│      /     /       /     /        /     /       /     /       /     /       /     /  │
│     /     /       /     /        /     /       /     /       /     /       /     /   │
│    /     /       /     /        /     / Green /     /       /     /       /     /    │
│   /     /       /     /        /     /       /     /       /     /       /     /     │
│  /     /       /     /        /     /       /     /       /     /       /     /      │
│ /     /       /     /        /     /       /     /       /     /       /     /       │
└──────────────────────────────────────────────────────────────────────────────────────┘
```

### Draw to texture 1:

In Slot N - 2 of texture 1, draw unclipped red rectangle and, in slot N
- 1, draw the unclipped
blue rectangle.

Slot N - 2:
```
┌──────────────────────────────────────────────────────────────────────────────────────┐
│      ──              ───                            ──              ───              │
│  ────            ────               ──          ────            ────               ──│
│──           ─────               ────          ──           ─────               ────  │
│        ─────                ────        Red           ─────                ────      │
│    ────                 ────                      ────                 ────          │
│ ───                 ────                       ───                 ────              │
│                  ───                                            ───                  │
└──────────────────────────────────────────────────────────────────────────────────────┘
```
Slot N - 1:
```
┌──────────────────────────────────────────────────────────────────────────────────────┐
│                                           ────────────────────────────────────────── │
│───────────────────────────────────────────                                           │
│                                                                                      │
│                                         Blue                          ───────────────│
│                                           ────────────────────────────               │
│               ────────────────────────────                                           │
│───────────────                                                                       │
└──────────────────────────────────────────────────────────────────────────────────────┘
```

### Round 1

At this point, we have three slots that contain our unclipped
rectangles. In this round,
we start to sample those pixels to apply clipping (texture 1 samples
from texture 0 and
the render target view samples from texture 1).

### Draw to texture 0:

Slot N - 1 of texture 0 contains our unclipped green rectangle. In this
draw, we sample
the pixels from slot N - 2 from texture 1 to draw the blue rectangle
into this slot.

Slot N - 1:
```
┌─────────────────────────────────────────────────────────┌─────────────────────────────
│        /     /       /     /       /     /       /     /│        ──────────────      │
│       /     /       /     /       /     /       /     / │────────                    │
│      /     /       /     /       /     /       /     /  │                            │
│     /     /       /  Green      /     /       /     /   │           Blue             │
│    /     /       /     /       /     /       /     /    │                     ───────│
│   /     /       /     /       /     /       /     /     │       ──────────────       │
│  /     /       /     /       /     /       /     /      │───────                     │
└─────────────────────────────────────────────────────────└────────────────────────────┘
```

### Draw to texture 1:

Then, into Slot N - 2 of texture 1, which contains our red rectangle, we
sample the pixels
from slot N - 1 of texture 0 which contain our green and blue
rectangles.
```

┌────────────────────────────┌────────────────────────────┌─────────────────────────────
│      ──              ───   │       /     /       /     /│        ──────────────      │
│  ────            ────      │      /     /       /     / │────────                    │
│──           ─────          │     /     /       /     /  │                            │
│        ───Red              │    /     /Green  /     /   │           Blue             │
│    ────                ──  │   /     /       /     /    │                     ───────│
│ ───                ────    │  /     /       /     /     │       ──────────────       │
│                  ──        │ /     /       /     /      │───────                     │
└────────────────────────────└────────────────────────────└────────────────────────────┘
```

### Draw to render target

At this point, we can sample the pixels from slot N - 1 of texture 1 to
draw the final
rendition.

## Nuances

- When there are no clip/blend regions, we can render directly to the
final target.
- The above example provides an intuitive explanation for how rounds
after 3 clip depths
are scheduled. At clip depths 1 and 2, we can draw directly to the final
target within a
  single round.
- Before drawing into any slot, we need to clear it. If all slots can be
cleared or are free,
we can use a `LoadOp::Clear` operation. Otherwise, we need to clear the
dirty slots using
  a fine grained render pass.

For more information about this algorithm, see this [Zulip thread].

[Zulip thread]:
https://xi.zulipchat.com/#narrow/channel/197075-vello/topic/Spatiotemporal.20allocation.20.28hybrid.29/near/513442829

---------

Co-authored-by: Raph Levien <[email protected]>

Author: taj-p

Cargo.lock

@@ -150,8 +150,7 @@ pub fn vello_test(attr: TokenStream, item: TokenStream) -> TokenStream {
 
     // These tests currently don't work with `vello_hybrid`.
     skip_hybrid |= {
-        input_fn_name_str.contains("clip")
-            || input_fn_name_str.contains("compose")
+        input_fn_name_str.contains("compose")
             || input_fn_name_str.contains("gradient")
             || input_fn_name_str.contains("image")
             || input_fn_name_str.contains("layer")

sparse_strips/vello_dev_macros/src/lib.rs

@@ -15,8 +15,9 @@ publish = false
 workspace = true
 
 [dependencies]
-vello_common = { workspace = true }
 bytemuck = { workspace = true, features = ["derive"] }
+thiserror = { workspace = true }
+vello_common = { workspace = true }
 wgpu = { workspace = true }
 
 [dev-dependencies]

sparse_strips/vello_hybrid/Cargo.toml

@@ -11,7 +11,6 @@ use vello_common::kurbo::{Affine, Stroke};
 use vello_common::pico_svg::{Item, PicoSvg};
 use vello_common::pixmap::Pixmap;
 use vello_hybrid::{DimensionConstraints, Scene};
-use wgpu::RenderPassDescriptor;
 
 /// Main entry point for the headless rendering example.
 /// Takes two command line arguments:
@@ -91,28 +90,20 @@ async fn run() {
         width: width.into(),
         height: height.into(),
     };
-    renderer.prepare(&device, &queue, &scene, &render_size);
     // Copy texture to buffer
     let mut encoder = device.create_command_encoder(&wgpu::CommandEncoderDescriptor {
         label: Some("Vello Render To Buffer"),
     });
-    {
-        let mut pass = encoder.begin_render_pass(&RenderPassDescriptor {
-            label: Some("Render Pass"),
-            color_attachments: &[Some(wgpu::RenderPassColorAttachment {
-                view: &texture_view,
-                resolve_target: None,
-                ops: wgpu::Operations {
-                    load: wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
-                    store: wgpu::StoreOp::Store,
-                },
-            })],
-            depth_stencil_attachment: None,
-            occlusion_query_set: None,
-            timestamp_writes: None,
-        });
-        renderer.render(&scene, &mut pass);
-    }
+    renderer
+        .render(
+            &scene,
+            &device,
+            &queue,
+            &mut encoder,
+            &render_size,
+            &texture_view,
+        )
+        .unwrap();
 
     // Create a buffer to copy the texture data
     let bytes_per_row = (u32::from(width) * 4).next_multiple_of(256);

sparse_strips/vello_hybrid/examples/render_to_file.rs

@@ -170,13 +170,6 @@ impl AppState {
             height: self.height,
         };
 
-        self.renderer_wrapper.renderer.prepare(
-            &self.renderer_wrapper.device,
-            &self.renderer_wrapper.queue,
-            &self.scene,
-            &render_size,
-        );
-
         let surface_texture = self.renderer_wrapper.surface.get_current_texture().unwrap();
         let surface_texture_view = surface_texture
             .texture
@@ -186,26 +179,18 @@ impl AppState {
             .renderer_wrapper
             .device
             .create_command_encoder(&wgpu::CommandEncoderDescriptor { label: None });
-        {
-            let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
-                label: None,
-                color_attachments: &[Some(wgpu::RenderPassColorAttachment {
-                    view: &surface_texture_view,
-                    resolve_target: None,
-                    ops: wgpu::Operations {
-                        load: wgpu::LoadOp::Clear(wgpu::Color::BLACK),
-                        store: wgpu::StoreOp::Store,
-                    },
-                })],
-                depth_stencil_attachment: None,
-                occlusion_query_set: None,
-                timestamp_writes: None,
-            });
-
-            self.renderer_wrapper
-                .renderer
-                .render(&self.scene, &mut pass);
-        }
+
+        self.renderer_wrapper
+            .renderer
+            .render(
+                &self.scene,
+                &self.renderer_wrapper.device,
+                &self.renderer_wrapper.queue,
+                &mut encoder,
+                &render_size,
+                &surface_texture_view,
+            )
+            .unwrap();
 
         self.renderer_wrapper.queue.submit([encoder.finish()]);
         surface_texture.present();
@@ -504,32 +489,24 @@ pub async fn render_scene(scene: vello_hybrid::Scene, width: u16, height: u16) {
         width: width as u32,
         height: height as u32,
     };
-    renderer.prepare(&device, &queue, &scene, &render_size);
-
     let surface_texture = surface.get_current_texture().unwrap();
     let surface_texture_view = surface_texture
         .texture
         .create_view(&wgpu::TextureViewDescriptor::default());
 
     let mut encoder =
         device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: None });
-    {
-        let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
-            label: None,
-            color_attachments: &[Some(wgpu::RenderPassColorAttachment {
-                view: &surface_texture_view,
-                resolve_target: None,
-                ops: wgpu::Operations {
-                    load: wgpu::LoadOp::Clear(wgpu::Color::BLACK),
-                    store: wgpu::StoreOp::Store,
-                },
-            })],
-            depth_stencil_attachment: None,
-            occlusion_query_set: None,
-            timestamp_writes: None,
-        });
-        renderer.render(&scene, &mut pass);
-    }
+
+    renderer
+        .render(
+            &scene,
+            &device,
+            &queue,
+            &mut encoder,
+            &render_size,
+            &surface_texture_view,
+        )
+        .unwrap();
 
     queue.submit([encoder.finish()]);
     surface_texture.present();

sparse_strips/vello_hybrid/examples/webgl/src/lib.rs

@@ -13,7 +13,6 @@ use vello_common::color::{AlphaColor, Srgb};
 use vello_common::kurbo::{Affine, Vec2};
 use vello_hybrid::{RenderSize, Renderer, Scene};
 use vello_hybrid_scenes::{AnyScene, get_example_scenes};
-use wgpu::RenderPassDescriptor;
 use winit::{
     application::ApplicationHandler,
     event::{ElementState, KeyEvent, MouseButton, MouseScrollDelta, WindowEvent},
@@ -271,12 +270,6 @@ impl ApplicationHandler for App<'_> {
                     width: surface.config.width,
                     height: surface.config.height,
                 };
-                self.renderers[surface.dev_id].as_mut().unwrap().prepare(
-                    &device_handle.device,
-                    &device_handle.queue,
-                    &self.scene,
-                    &render_size,
-                );
 
                 let surface_texture = surface
                     .surface
@@ -293,26 +286,18 @@ impl ApplicationHandler for App<'_> {
                         .create_command_encoder(&wgpu::CommandEncoderDescriptor {
                             label: Some("Vello Render to Surface pass"),
                         });
-                {
-                    let mut pass = encoder.begin_render_pass(&RenderPassDescriptor {
-                        label: Some("Render to Texture Pass"),
-                        color_attachments: &[Some(wgpu::RenderPassColorAttachment {
-                            view: &texture_view,
-                            resolve_target: None,
-                            ops: wgpu::Operations {
-                                load: wgpu::LoadOp::Clear(wgpu::Color::BLACK),
-                                store: wgpu::StoreOp::Store,
-                            },
-                        })],
-                        depth_stencil_attachment: None,
-                        occlusion_query_set: None,
-                        timestamp_writes: None,
-                    });
-                    self.renderers[surface.dev_id]
-                        .as_mut()
-                        .unwrap()
-                        .render(&self.scene, &mut pass);
-                }
+                self.renderers[surface.dev_id]
+                    .as_mut()
+                    .unwrap()
+                    .render(
+                        &self.scene,
+                        &device_handle.device,
+                        &device_handle.queue,
+                        &mut encoder,
+                        &render_size,
+                        &texture_view,
+                    )
+                    .unwrap();
 
                 device_handle.queue.submit([encoder.finish()]);
                 surface_texture.present();

sparse_strips/vello_hybrid/examples/winit/src/main.rs

@@ -0,0 +1,51 @@
+// Copyright 2025 the Vello Authors
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+// This shader clears specific slots in slot textures to transparent pixels.
+
+// Assumes this texture consists of a single column of slots of `config.slot_height`, 
+// numbering from 0 to `texture_height / slot_height - 1` from top to bottom.
+
+struct Config {
+    // Width of a slot (matching `WideTile::WIDTH` and the width of a slot texture).
+    slot_width: u32,
+    // Height of a slot (matching `Tile::HEIGHT`)
+    slot_height: u32,
+    // Total height of the texture (slot_height * number_of_slots)
+    texture_height: u32,
+    // Padding for 16-byte alignment
+    _padding: u32,
+}
+
+@group(0) @binding(0)
+var<uniform> config: Config;
+
+@vertex
+fn vs_main(
+    @builtin(vertex_index) vertex_index: u32,
+    @location(0) index: u32,
+) -> @builtin(position) vec4<f32> {
+    // Map vertex_index (0-3) to quad corners:
+    // 0 → (0,0), 1 → (1,0), 2 → (0,1), 3 → (1,1)
+    let x = f32(vertex_index & 1u);
+    let y = f32(vertex_index >> 1u);
+    
+    // Calculate the y-position based on the slot index
+    let slot_y_offset = f32(index * config.slot_height);
+    
+    // Scale to match slot dimensions
+    let pix_x = x * f32(config.slot_width);
+    let pix_y = slot_y_offset + y * f32(config.slot_height);
+    
+    // Convert to NDC
+    let ndc_x = pix_x * 2.0 / f32(config.slot_width) - 1.0;
+    let ndc_y = 1.0 - pix_y * 2.0 / f32(config.texture_height);
+    
+    return vec4<f32>(ndc_x, ndc_y, 0.0, 1.0);
+}
+
+@fragment
+fn fs_main(@builtin(position) position: vec4<f32>) -> @location(0) vec4<f32> {
+    // Clear with transparent pixels
+    return vec4<f32>(0.0, 0.0, 0.0, 0.0);
+} 

sparse_strips/vello_hybrid/shaders/clear_slots.wgsl

@@ -9,13 +9,19 @@
 //
 // The alpha values are stored in a texture and sampled during fragment shading.
 // This approach optimizes memory usage by only storing alpha data where needed.
+//
+// The `StripInstance`'s `rgba_or_slot` field can either encode a color or a slot index.
+// If the alpha value is non-zero, the fragment shader samples the alpha texture.
+// Otherwise, the fragment shader samples the source clip texture using the given slot index.
 
 struct Config {
-    // Width of the rendering target    
+    // Width of the rendering target
     width: u32,
     // Height of the rendering target
     height: u32,
     // Height of a strip in the rendering
+    // CAUTION: When changing this value, you must also update the fragment shader's
+    // logic to handle the new strip height.
     strip_height: u32,
     // Number of trailing zeros in alphas_tex_width (log2 of width).
     // Pre-calculated on CPU since WebGL2 doesn't support `firstTrailingBit`.
@@ -29,17 +35,17 @@ struct StripInstance {
     @location(1) widths: u32,
     // Alpha texture column index where this strip's alpha values begin
     @location(2) col: u32,
-    // [r, g, b, a] packed as u8's
-    @location(3) rgba: u32,
+    // [r, g, b, a] packed as u8's or a slot index when alpha is 0
+    @location(3) rgba_or_slot: u32,
 }
 
 struct VertexOutput {
-    // Texture coordinates for the current fragment 
+    // Texture coordinates for the current fragment
     @location(0) tex_coord: vec2<f32>,
     // Ending x-position of the dense (alpha) region
     @location(1) @interpolate(flat) dense_end: u32,
-    // RGBA color value
-    @location(2) @interpolate(flat) color: u32,
+    // Color value or slot index when alpha is 0
+    @location(2) @interpolate(flat) rgba_or_slot: u32,
     // Normalized device coordinates (NDC) for the current vertex
     @builtin(position) position: vec4<f32>,
 };
@@ -77,21 +83,22 @@ fn vs_main(
 
     out.position = vec4<f32>(ndc_x, ndc_y, 0.0, 1.0);
     out.tex_coord = vec2<f32>(f32(instance.col) + x * f32(width), y * f32(config.strip_height));
-    out.color = instance.rgba;
+    out.rgba_or_slot = instance.rgba_or_slot;
     return out;
 }
 
 @group(0) @binding(0)
 var alphas_texture: texture_2d<u32>;
 
+@group(0) @binding(2)
+var clip_input_texture: texture_2d<f32>;
+
 @fragment
 fn fs_main(in: VertexOutput) -> @location(0) vec4<f32> {
     let x = u32(floor(in.tex_coord.x));
     var alpha = 1.0;
     // Determine if the current fragment is within the dense (alpha) region
     // If so, sample the alpha value from the texture; otherwise, alpha remains fully opaque (1.0)
-    // TODO: This is a branch, but we can make it branchless by using a select
-    // would it be faster to do a texture lookup for every pixel?
     if x < in.dense_end {
         let y = u32(floor(in.tex_coord.y));
         // Retrieve alpha value from the texture. We store 16 1-byte alpha
@@ -108,18 +115,28 @@ fn fs_main(in: VertexOutput) -> @location(0) vec4<f32> {
         let channel_index = alphas_index % 4u;
         // Calculate texel coordinates
         let tex_x = texel_index & (alphas_tex_width - 1u);
-        let tex_y = texel_index >> config.alphas_tex_width_bits;                  
-        
+        let tex_y = texel_index >> config.alphas_tex_width_bits;
+
         // Load all 4 channels from the texture
         let rgba_values = textureLoad(alphas_texture, vec2<u32>(tex_x, tex_y), 0);
-        
+
         // Get the column's alphas from the appropriate RGBA channel based on the index
         let alphas_u32 = unpack_alphas_from_channel(rgba_values, channel_index);
         // Extract the alpha value for the current y-position from the packed u32 data
         alpha = f32((alphas_u32 >> (y * 8u)) & 0xffu) * (1.0 / 255.0);
     }
-    // Apply the alpha value to the unpacked RGBA color
-    return alpha * unpack4x8unorm(in.color);
+    // Apply the alpha value to the unpacked RGBA color or slot index
+    let alpha_byte = in.rgba_or_slot >> 24u;
+    if alpha_byte != 0 {
+        // in.rgba_or_slot encodes a color
+        return alpha * unpack4x8unorm(in.rgba_or_slot);
+    } else {
+        // in.rgba_or_slot encodes a slot in the source clip texture
+        let clip_x = u32(in.position.x) & 0xFFu;
+        let clip_y = (u32(in.position.y) & 3) + in.rgba_or_slot * config.strip_height;
+        let clip_in_color = textureLoad(clip_input_texture, vec2(clip_x, clip_y), 0);
+        return alpha * clip_in_color;
+    }
 }
 
 fn unpack_alphas_from_channel(rgba: vec4<u32>, channel_index: u32) -> u32 {
@@ -136,6 +153,7 @@ fn unpack_alphas_from_channel(rgba: vec4<u32>, channel_index: u32) -> u32 {
 // Polyfills `unpack4x8unorm`.
 //
 // Downlevel targets do not support native WGSL `unpack4x8unorm`.
+// TODO: Remove once we upgrade to WGPU 25.
 fn unpack4x8unorm(rgba_packed: u32) -> vec4<f32> {
     // Extract each byte and convert to float in range [0,1]
     return vec4<f32>(