Skip to content

Commit 6b05040

Browse files
atlv24atlv24
authored
Merge branch 'trunk' into ad/bary-no-persp
2 parents 4074655 + 1d520a0 commit 6b05040

17 files changed

+1053
-42
lines changed

‎.claude/skills/cts-triage/SKILL.md‎

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
---
2+
name: cts-triage
3+
description: Run CTS test suites and investigate failures
4+
---
5+
6+
# Triage Process
7+
8+
When working on a category of CTS tests, follow this systematic process to identify issues, prioritize fixes, and document findings.
9+
10+
## Step 0: Divide Into Manageable Chunks
11+
12+
List all the tests matching a selector:
13+
14+
```bash
15+
cargo xtask cts -- --list 'webgpu:api,validation,*' 2>&1 | wc -l
16+
```
17+
18+
If there is a reasonable number of tests matching the selector (less than a
19+
couple hundred or so, but this isn't a hard cutoff), then you can proceed with
20+
triage. Otherwise, make a list of more detailed wildcards that match fewer
21+
tests, verify that each wildcard matches a reasonable number of tests, and
22+
triage each wildcard separately.
23+
24+
## Step 1: Get Overall Statistics
25+
26+
Run the full test suite for the category to understand the scope:
27+
28+
```bash
29+
cargo xtask cts 'webgpu:api,validation,category:*' 2>&1 | grep -E "(Summary|Passed|Failed)" | tail -5
30+
```
31+
32+
This gives you the pass rate and number of failures. Document this as your baseline.
33+
34+
## Step 2: Identify Test Subcategories
35+
36+
Review the output from the running the CTS (with or without `--list`) to
37+
identify any subcategories that may exist within the suite being analyzed.
38+
Subcategories typically have an additional `:`- or `,`-delimited word in the
39+
test name. Running tests by subcategory may be more manageable than running
40+
with the entire suite at once or running individual tests.
41+
42+
## Step 3: Run Each Subcategory
43+
44+
Test each subcategory individually to identify which ones are passing vs failing:
45+
46+
```bash
47+
cargo xtask cts 'webgpu:api,validation,category:subcategory:*' 2>&1 | grep -E "(Passed|Failed|Skipped)" | tail -3
48+
```
49+
50+
Track the pass rate for each subcategory. This helps you identify:
51+
- What's already working (don't break it!)
52+
- Where the failures are concentrated
53+
- Which issues affect multiple categories
54+
55+
## Step 4: Analyze Failure Patterns
56+
57+
For failing subcategories, look at what tests are failing:
58+
59+
```bash
60+
cargo xtask cts 'webgpu:api,validation,category:subcategory:*' 2>&1 | grep "\[fail\]" | head -20
61+
```
62+
63+
Look for patterns:
64+
- **Format-specific failures**: May indicate missing format capability checks
65+
- **Parameter-specific failures**: Validation missing for specific parameter combinations
66+
67+
## Step 5: Examine Specific Failures
68+
69+
Pick a representative failing test and run it individually to see the error:
70+
71+
```bash
72+
cargo xtask cts 'webgpu:api,validation,category:subcategory:specific_test' 2>&1 | tail -30
73+
```
74+
75+
Look for:
76+
- **"EXPECTATION FAILED: DID NOT REJECT"**: wgpu is accepting invalid input (validation gap)
77+
- **"Validation succeeded unexpectedly"**: Similar to above
78+
- **"Unexpected validation error occurred"**: wgpu is rejecting valid input
79+
- **Error message content**: Tells you what validation is triggering or missing
80+
81+
## Step 6: Check CTS Test Source
82+
83+
To understand what the test expects, read the TypeScript source:
84+
85+
```bash
86+
grep -A 40 "test_name" cts/src/webgpu/api/validation/path/file.spec.ts
87+
```
88+
89+
The test source shows:
90+
- What configurations are being tested
91+
- What the expected behavior is (pass/fail)
92+
- The validation rules from the WebGPU spec
93+
- Comments explaining the rationale
94+
95+
## Step 7: Categorize Issues
96+
97+
Group failures into categories:
98+
99+
**High Priority - Validation Gaps:**
100+
- wgpu accepts invalid configurations that should fail
101+
- Security or correctness implications
102+
- Example: Accepting wrong texture formats, missing aspect checks
103+
104+
**Medium Priority - Spec Compliance:**
105+
- Edge cases not handled correctly
106+
- Optional field validation issues
107+
- Example: depthCompare optional field handling
108+
109+
**Low Priority - Minor Gaps:**
110+
- Less common scenarios
111+
- Limited real-world impact
112+
- Example: Depth bias with non-triangle topologies
113+
114+
**Known Issues - Skip:**
115+
- Known failure patterns (documented in AGENTS.md)
116+
- Track count but don't try to fix
117+
118+
## Step 8: Identify Root Causes
119+
120+
For validation gaps, find where validation should happen:
121+
122+
1. **Search for existing validation:**
123+
```bash
124+
grep -n "relevant_keyword" wgpu-core/src/device/resource.rs
125+
```
126+
127+
2. **Look for render/compute pipeline creation:**
128+
- Render pipeline: `wgpu-core/src/device/resource.rs` around `create_render_pipeline`
129+
- Compute pipeline: Similar location
130+
- Look for existing validation patterns you can follow
131+
132+
3. **Check for helper functions:**
133+
```bash
134+
grep "fn is_" wgpu-types/src/texture/format.rs
135+
```
136+
137+
4. **Find error enums:**
138+
```bash
139+
grep "pub enum.*Error" wgpu-core/src/pipeline.rs
140+
```
141+
142+
## Step 9: Implement Fixes
143+
144+
When implementing fixes:
145+
146+
1. **Add error variants if needed** (in `wgpu-core/src/pipeline.rs`)
147+
2. **Add helper methods** (in `wgpu-types` if checking properties)
148+
3. **Add validation checks** (in `wgpu-core/src/device/resource.rs`)
149+
4. **Test the fix** with specific failing tests
150+
5. **Run full subcategory** to verify all related tests pass
151+
6. **Check you didn't break passing tests**
152+
153+
## Step 10: Document Findings
154+
155+
Create or update a triage document (e.g., `category_triage.md`).
156+
157+
Do not write information about changes you have made to the triage document. Only capture the state of the tests and any investigation into open issues.
158+
159+
````markdown
160+
# Category CTS Tests - Triage Report
161+
162+
**Overall Status:** XP/YF/ZS (%/%/%)
163+
164+
## Passing Sub-suites ✅
165+
[List sub-suites that have no failures (all pass or skip)]
166+
167+
## Remaining Issues ⚠️
168+
[List sub-suites that have failures and if it can be stated concisely, a summary of the issue]
169+
170+
## Issue Detail
171+
[List detail of any investigation into failures. Do not go into detail about passed suites, just list the failures.]
172+
173+
### 1. title, e.g. a distinguishing word from the test selector
174+
**Test selector:** `webgpu:api,validation,render_pipeline,depth_stencil_state:format:*`
175+
**What it tests:** [Description]
176+
**Example failure:**
177+
[a selector for a single failing test, e.g.:]
178+
```
179+
webgpu:api,validation,render_pipeline,depth_stencil_state:depthCompare_optional:isAsync=false;format="stencil8"
180+
```
181+
182+
**Error:**
183+
[error message from the failing tests, e.g.:]
184+
```
185+
Unexpected validation error occurred: Depth/stencil state is invalid:
186+
Format Stencil8 does not have a depth aspect, but depth test/write is enabled
187+
```
188+
189+
**Root cause:**
190+
[Your analysis of the root cause. Do not speculate. Only include the results of specific investigation you have done.]
191+
The validation is triggering incorrectly. When `depthCompare` is undefined/missing in JavaScript, it's getting translated to a default value that makes `is_depth_enabled()` return true, even for stencil-only formats.
192+
193+
**Fix needed:**
194+
[Your proposed fix. Again, do not speculate. Only state the fix if it is obvious from the root cause analysis, or if you have done specific investigation into how to fix it.]
195+
196+
### 2. title
197+
[repeat as needed for additional issues]
198+
````
199+
200+
## Step 11: Update test.lst
201+
202+
For fixed tests that are now passing, add them to `cts_runner/test.lst`:
203+
204+
1. **Use wildcards** to minimize lines:
205+
```
206+
webgpu:api,validation,category:subcategory:isAsync=false;*
207+
```
208+
209+
2. **Group related tests** when possible:
210+
- If multiple subcategories all pass, use a higher-level wildcard
211+
- Example: `webgpu:api,validation,category:*` if all subcategories pass
212+
213+
3. **Maintain alphabetical order** roughly in the file
214+
215+
## Step 12: Verify and Build
216+
217+
Before finishing:
218+
219+
```bash
220+
# Format code
221+
cargo fmt
222+
223+
# Check for errors
224+
cargo clippy --tests
225+
226+
# Build to ensure no compilation errors
227+
cargo build
228+
229+
# Run the tests you added to test.lst
230+
cargo xtask cts 'webgpu:api,validation,category:subcategory:isAsync=false;*'
231+
```
232+
233+
# Common Patterns
234+
235+
If the user asked to investigate a failure, and the cause of the failure is
236+
noted here with "do not attempt to fix", then stop and ask the user before
237+
attempting to fix.
238+
239+
**Pattern: Format-specific failures**
240+
- Check if format validation is missing
241+
- Look for `is_depth_stencil_format()`, `is_color_format()` etc.
242+
- May need to add format capability checks
243+
244+
**Pattern: Aspect-related failures**
245+
- Check if code validates format aspects (DEPTH, STENCIL, COLOR)
246+
- Use `hal::FormatAspects::from(format)` to check
247+
- Validate operations match available aspects
248+
249+
**Pattern: Optional field failures**
250+
- May be WebGPU optional field semantics issue
251+
- Check if undefined in JS becomes a default value in Rust
252+
- May need to distinguish "not set" from "set to default"
253+
254+
**Pattern: Atomics accepted incorrectly**
255+
- Naga allows referencing an atomic directly in an expression
256+
- Should only allow accessing via `atomicLoad`, `atomicStore`, etc.
257+
- Only investigate as necessary to confirm this is the issue. Do not attempt to fix. Refer user to https://github.com/gfx-rs/wgpu/issues/5474.
258+
259+
**Pattern: Error reporting for destroyed resources**
260+
- Tests that check for validation errors when a destroyed resource is used. `wgpu` often reports these errors later than WebGPU requires, causing the tests to fail.
261+
- `wgpu` may report these errors earlier than it should, causing the test to fail with an unexpected validation error.
262+
- Look for:
263+
- Tests with `state="destroyed"` parameter
264+
- Tests checking that operations on destroyed buffers or textures should fail
265+
- Example failing tests:
266+
- `webgpu:api,validation,encoding,cmds,compute_pass:indirect_dispatch_buffer_state:` with `state="destroyed"` subcases
267+
- Only investigate as necessary to confirm this is the issue. Do not attempt to fix. Refer user to https://github.com/gfx-rs/wgpu/issues/7881.
268+
269+
# Tips
270+
271+
- **Start with high-impact fixes**: Validation gaps with security implications
272+
- **Look for existing patterns**: Other validation code shows the style
273+
- **Test incrementally**: Fix one category at a time, verify it works
274+
- **Document as you go**: Don't wait until the end to write the triage
275+
- **Ask for clarification**: If test expectations are unclear, check the WebGPU spec or test source
276+
- **Track your progress**: Update pass rates as you fix issues

‎.claude/skills/webgpu-specs/SKILL.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
name: webgpu-specs
3+
description: Download WebGPU and WGSL specifications for use as a reference
4+
allowed-tools: "Bash(sh .claude/skills/webgpu-specs/download.sh)"
5+
---
6+
7+
Run `sh .claude/skills/webgpu-specs/download.sh` to download the
8+
WebGPU and WGSL specifications if they are not present or if they have
9+
been updated. You do not need to change directory before running the script.
10+
11+
After the specs are downloaded, you can search in `target/claude/webgpu-spec.bs`
12+
and `target/claude/wgsl-spec.bs` for relevant sections of the specification.
13+
14+
When referencing the specifications, prefer to use named anchors rather than
15+
line numbers. For example, to reference the "Object Descriptors" section, which has the
16+
following header:
17+
18+
```
19+
### Object Descriptors ### {#object-descriptors}
20+
```
21+
22+
Use the URL <https://gpuweb.github.io/gpuweb/#object-descriptors> so the user
23+
can click to navigate directly to that section.
24+
25+
For the WGSL specification, the base URL is <https://gpuweb.github.io/gpuweb/wgsl/>.
26+
27+
If necessary, read additional content from the file to find the header preceding
28+
the text you want to reference. You may provide line numbers as additional
29+
context, but always make every effort to provide the user with a clickable link.

‎.claude/skills/webgpu-specs/download.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/bin/sh
2+
3+
set -e
4+
5+
TARGET_DIR="$(cargo metadata --format-version 1 | jq -r ".target_directory")"
6+
7+
WEBGPU="$TARGET_DIR/claude/webgpu-spec"
8+
WGSL="$TARGET_DIR/claude/wgsl-spec"
9+
mkdir -p "$TARGET_DIR/claude"
10+
11+
if [ -f "$WEBGPU.etag" ]; then
12+
curl --etag-save "$WEBGPU.etag.new" --etag-compare "$WEBGPU.etag" -fsSL https://raw.githubusercontent.com/gpuweb/gpuweb/main/spec/index.bs -o "$WEBGPU.bs"
13+
[ -s "$WEBGPU.etag.new" ] && mv "$WEBGPU.etag.new" "$WEBGPU.etag" || rm "$WEBGPU.etag.new"
14+
else
15+
curl --etag-save "$WEBGPU.etag" https://raw.githubusercontent.com/gpuweb/gpuweb/main/spec/index.bs -o "$WEBGPU.bs"
16+
fi
17+
18+
if [ -f "$WGSL.etag" ]; then
19+
curl --etag-save "$WGSL.etag.new" --etag-compare "$WGSL.etag" -fsSL https://raw.githubusercontent.com/gpuweb/gpuweb/main/wgsl/index.bs -o "$WGSL.bs"
20+
[ -s "$WGSL.etag.new" ] && mv "$WGSL.etag.new" "$WGSL.etag" || rm "$WGSL.etag.new"
21+
else
22+
curl --etag-save "$WGSL.etag" https://raw.githubusercontent.com/gpuweb/gpuweb/main/wgsl/index.bs -o "$WGSL.bs"
23+
fi

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,6 @@ wgpu/red.png
4444

4545
# Temporary clone location for wasm-bindgen mirroring
4646
wgpu/src/backend/webgpu/webgpu_sys/wasm_bindgen_clone_tmp
47+
48+
# LLM-related
49+
.claude/settings.local.json

0 commit comments

Comments
 (0)