Extend screenshot tests with web backend coverage#3
Draft
zzuegg wants to merge 16 commits into
Draft
Conversation
Restructure screenshot tests to support full cross-backend coverage including browser-based WebGPU testing: - scenes/ — shared scene definitions, discovery, tolerances (25 scenes) - desktop/ — existing GL/VK/WebGPU-native runner (unchanged behavior) - web/ — new browser runner using headless Chrome + CDP (zero deps) The web runner compiles test scenes to JS via TeaVM, serves them from an embedded HTTP server, and captures canvas pixels via Chrome DevTools Protocol over JDK's built-in WebSocket. No Playwright/Selenium needed. Also adds GitHub Actions workflow for web screenshot tests with Chrome. https://claude.ai/code/session_019VNC29wPnAedL8EyyU15XY
- Use configurations.all { exclude... } matching platforms/web pattern
- Add explicit ui dependency for scene classes
- Remove logback (incompatible with SLF4J exclusion)
- Add --stacktrace to CI steps for better error visibility
- Split CI into compile + build steps for easier debugging
https://claude.ai/code/session_019VNC29wPnAedL8EyyU15XY
Split assembleWebTest into compileJava → generateJavaScript → assemble to isolate whether the failure is Java compilation or TeaVM JS generation. Capture TeaVM log as uploadable artifact. https://claude.ai/code/session_019VNC29wPnAedL8EyyU15XY
Use actions/github-script to post the last 80 lines of the build log as a PR comment on failure, so we can see the actual error. https://claude.ai/code/session_019VNC29wPnAedL8EyyU15XY
- Fix circular Gradle dependency by replacing platforms:web dep with inline TestWebPlatform (both modules use TeaVM plugin) - Fix CDP client: connect to page target (not browser), fix shared pending map between WebSocket listener and send() - Fix TeaVM compatibility: replace ConcurrentLinkedQueue/CountDownLatch in engine with TeaVM-compatible alternatives (synchronized ArrayDeque, simple counter) - Force-load NativeStruct generated classes for TeaVM (can't use Class.forName dynamic loading) - Make debug UI init graceful (falls back to disabled on TeaVM) - Add start/ack synchronization protocol between test runner and app - Add error signaling for scenes that crash in browser - Fix Slang WASM path (slang/ subdirectory) and JS script path (js/) - Add SaveWebReferences task for generating reference screenshots - Add 12 initial reference screenshots for webgpu-browser - Update CI workflow with test results upload and summary - Update screenshot test docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add webgpu-browser backend to report generator's backend list - Add mergeWebScreenshots() to merge web screenshots from separate dir - Pass web screenshot dir as 4th arg in generateReport Gradle task - Support absolute paths for cross-directory image references Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Implement queueWriteTexture in TeaVmWgpuBindings using queue.writeTexture() JS API with byte[] data transfer - Simplify SaveWebReferences to copy from test captures instead of running a separate Chrome session - Add all 30 reference screenshots (previously only 12 — texture and mixed material scenes were failing due to missing texture upload) All 30 web screenshot tests now pass with reference comparison. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generates a visual grid comparing captured vs reference screenshots. Run: ./gradlew :samples:tests:screenshot:web:generateReport Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gradle 9.x resolves :platforms:desktop to :samples:tests:screenshot:desktop when both share the 'desktop' leaf name. Renaming to screenshot-desktop and screenshot-web avoids the collision. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OffscreenCanvas drawImage approach returned all-black pixels because WebGPU's swap chain texture is invalidated after presentation. Using CDP's Page.captureScreenshot captures the actual composited page content. Regenerated all 30 reference screenshots with real rendered content. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After the submodule split, scenes are in a separate JAR. The file-based classpath scanner couldn't find them, resulting in 0 dynamic tests. Added JAR entry scanning via JarURLConnection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use Emulation.setDeviceMetricsOverride to set viewport to exactly 256x256 with DPR=1, eliminating black borders in Page.captureScreenshot - Remove broken canvas drawImage approach (WebGPU swap chain clears before readback in headless Chrome regardless of timing) - Screenshots now contain 100% rendered content - Regenerated all 30 reference screenshots Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oss-check - DebugUiOverlay loads shader via ShaderManager.loadShaderFile() instead of getClassLoader().getResourceAsStream(), enabling web/TeaVM support - Made ShaderManager.loadShaderFile() public - Added cross-backend comparison: browser WebGPU vs native WebGPU with Tolerance.crossPlatform() (soft warning, not hard failure) - Fixed viewport: Emulation.setDeviceMetricsOverride for full-frame capture - Added desktop webgpu references to web test classpath for cross-check - Regenerated all 30 references with correct full-frame content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Launch Chrome with about:blank, set Emulation.setDeviceMetricsOverride to 256x256 DPR=1, THEN navigate to app — ensures canvas fills viewport - Use Page.captureScreenshot (readCanvasPixels returns black in headless Chrome because WebGPU canvas drawImage is not supported) - Navigate via Page.navigate + re-enable Runtime after context reset - ShaderManager.loadShaderFile: add classpath fallback for desktop - Remove unused snapshotCanvas/RAF code from WebTestApp - Regenerated all 30 references with full-frame content Desktop debug UI renders correctly (8 colors). Browser debug UI shows empty scene — NkBuiltinFont requires native resources not in TeaVM. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scenes(shared),desktop(GL/VK/WebGPU-native),web(browser WebGPU)Test plan
scenesmodule compiles (shared scene definitions)desktopmodule tests pass (existing GL/VK/WebGPU-native tests, unchanged behavior)webmodule TeaVM compilation succeedshttps://claude.ai/code/session_019VNC29wPnAedL8EyyU15XY