feat(test): local Android Game Loop harness with golden screenshots#2445
Open
leanmendoza wants to merge 3 commits into
Open
feat(test): local Android Game Loop harness with golden screenshots#2445leanmendoza wants to merge 3 commits into
leanmendoza wants to merge 3 commits into
Conversation
Adds an on-device E2E test harness driven by the Firebase Test Lab Android Game Loop contract (com.google.intent.action.TEST_LOOP + integer scenario extra), runnable locally over adb with `cargo run -- game-loop` against a USB-connected device — no Test Lab / Blaze needed. xtask driver (src/game_loop/): install+pm clear, fire the TEST_LOOP intent, tail logcat for the RESULT line, pull user://gameloop_*.png, and compare vs goldens (tests/snapshots/gameloop/) or --bless them. Split by responsibility: options/adb/launch/screenshots/golden/scenario, each <200 lines. `--scenario all` fans out over the registry. GDScript runner (godot/src/test/game_loop/): autoload detects the Game Loop launch, runs the requested scenario, prints a machine-parseable RESULT to logcat, finishes via native gameLoopFinish. Typed result classes (GameLoopResult/CaptureResult), a GameLoopScenario base, and a shared LoginFlowScenario. Scenarios: 1 guest login, 2 screenshot, 3 login flow, 4 fresh-guest onboarding (random seed -> AVATAR_CREATE/ NAMING/DISCOVER_FTUE), 5 in-world load of aesironline.dcl.eth with async_wait_until_scene_loaded(30s). Deterministic captures (capture_mode.gd): momentary global freeze (scene_runner pause + avatars to canonical idle + time_scale=0 + canonicalized AnimationPlayer phase), optional live-world hide so the non-deterministic 3D scene/UI never draws, and manual per-golden binary masks (<golden>_mask.png) to exclude dynamic regions from the compare. Golden mode: a per-run guest-seed forces a deterministic guest via a seed-derived anchor through the STANDARD Play-as-Guest flow, reachable ONLY through this harness (no deeplink/UI path). Fix image_comparison: normalize per-pixel distance by 255*sqrt(channels) (was sqrt(255*3), ~18x too small, produced negative similarities) and clamp to [0,1]; add compare_images_similarity_masked; re-map caller thresholds via an exact monotonic transform to preserve test strictness.
Contributor
📦 Build Report🤖 Android
Build Status: ✅ Success 🍏 iOS
🔗 Workflow Run: View logs 🔄 Updated: 2026-07-15 13:44:11 UTC |
kuruk-mm
self-requested a review
July 14, 2026 14:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an on-device E2E test harness driven by the Firebase Test Lab Android Game Loop contract (
com.google.intent.action.TEST_LOOP+ integerscenarioextra), runnable locally over adb with a single command — no Test Lab / Blaze required:The device stays fixed, so golden screenshots are pixel-stable run to run (the whole reason to use a real device instead of Test Lab's rotating farm for regression goldens).
How it's structured
xtask driver (
src/game_loop/) — reproduces exactly what Test Lab does overadb: install +pm clear, fire theTEST_LOOPintent, tail logcat for theRESULTline, pulluser://gameloop_*.png, compare vstests/snapshots/gameloop/(or--bless). Split by responsibility —options/adb/launch/screenshots/golden/scenario, each < 200 lines.--scenario allfans out over the registry.GDScript runner (
godot/src/test/game_loop/) — an autoload detects the Game Loop launch, runs the requested scenario, prints a machine-parseableRESULTto logcat, and finishes via the nativegameLoopFinish. Typed result classes (GameLoopResult/CaptureResult), aGameLoopScenariobase, and a sharedLoginFlowScenario.AVATAR_CREATE/NAMING/DISCOVER_FTUEaesironline.dcl.ethwithasync_wait_until_scene_loaded(30s)Deterministic captures (
capture_mode.gd) — a momentary global freeze:scene_runnerpause + avatars to canonical idle +time_scale=0+ canonicalizedAnimationPlayerphase. Optional live-world hide so the non-deterministic 3D scene / SDK UI never draws, and manual per-golden binary masks (<golden>_mask.png) to exclude dynamic regions (place cards, random avatar/name) from the compare.Golden mode — a per-run
guest-seedforces a deterministic guest via a seed-derived anchor through the standard Play-as-Guest flow, reachable only through this harness (no deeplink/UI path, so a production install can't be driven into a forced identity).Comparator fix
compare_images_similaritynormalized per-pixel distance bysqrt(255*3)(~18× too small), which produced negative similarities. Fixed to255*sqrt(channels)+ clamp to[0,1], addedcompare_images_similarity_masked, and re-mapped caller thresholds via an exact monotonic transform so existing snapshot tests keep their strictness.Verified
Developed and verified end-to-end on a physical device (Samsung A54). Scenarios 1–5 run, pull shots, and compare; scenario 3 goldens committed.
Follow-ups (not in this PR)
_mask.pngfiles for scenario 4's dynamic screens (random avatar/name) and bless its goldens._mask.pngor--threshold 0.98— not the default 0.995.🤖 Generated with Claude Code