QVAC-18790 infra: scope e2e bootstrap to deps required by filtered tests#1991
Merged
Conversation
Victor-Rodzko
commented
May 12, 2026
c891ea9 to
1d8cb13
Compare
NamelsKing
previously approved these changes
May 12, 2026
Tier-based Approval Status |
lauripiisang
previously approved these changes
May 12, 2026
opaninakuffo
previously approved these changes
May 12, 2026
Wire up the new producer-driven bootstrap contract from @tetherto/qvac-test-suite 0.6.3 so the SDK e2e harness stops pre-loading every registered model when a --filter / --suite is in play. - ResourceManager.downloadAllOnce now accepts an optional `allowedDeps: ReadonlySet<string>` to scope downloads + pre-loads to a subset of registered models; without it the manager keeps the legacy "warm everything" behavior. - New `collect-test-deps` helper extracts the metadata.dependency / metadata.dependencies keys from a TestDefinition[] and dedupes them. - desktop/mobile bootstrap callbacks now take the `filteredTests?: TestDefinition[]` argument the framework supplies via register-ack, fed through collectTestDeps into downloadAllOnce. Locally cuts iOS bootstrap from ~25min to ~20s and desktop from ~10min (warm cache) to ~10s for `--filter transcription,parakeet`. Bumps @tetherto/qvac-test-suite to a temporary preview build of qvac-test-suite#70 (0.6.3-tmp.pr-70.runid-...); will be moved to ^0.6.3 once that PR lands and a real release is cut. Co-authored-by: Cursor <cursoragent@cursor.com>
b9d4a0c
1d8cb13 to
b9d4a0c
Compare
opaninakuffo
approved these changes
May 13, 2026
NamelsKing
approved these changes
May 13, 2026
Contributor
Author
|
/review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 What problem does this PR solve?
packages/sdk/tests-qvac) callResourceManager.downloadAllOnceon bootstrap and warm every registered model (~24 desktop / ~22 mobile) regardless of which tests will actually run.❌ No consumers connected within 1200s timeoutwhile the device was still pre-loading models.--filter registryneeds0,--filter translationneeds8).See QVAC-18790 for the full problem statement and CI evidence.
📝 How does it solve it?
tetherto/qvac-test-suite#70delivers the producer's post-filterTestDefinition[]to the consumer'sbootstrapcallback over MQTT (register-ack.filteredTestIds).ResourceManager.downloadAllOncegains an optionalallowedDeps: ReadonlySet<string>— when provided, downloads + pre-loads are scoped to that subset; when omitted, the legacy "warm everything" path is preserved (used by the CI cache-warmer step and any older framework).shared/collect-test-deps.tshelper extractsmetadata.dependency/metadata.dependenciesfrom the filtered tests (drops the"none"sentinel, dedupes).desktop/consumer.tsandmobile/consumer.tsbootstrapcallbacks now acceptfilteredTests?: TestDefinition[], deriveallowedDepsviacollectTestDeps, and pass it through.undefinedkeeps today's behaviour, so older framework versions and the unfiltered CI cache-warmer continue to populate the full cache.🧪 How was it tested?
Local
npx qvac-test run:local:desktop --filter transcription,parakeet→🎯 Bootstrap dep-filter active: keeping 4 dep(s); 20 of 24 defined excluded, bootstrap 9.97 s (was minutes).npx qvac-test run:local:ios --filter transcription,parakeet→keeping 4 dep(s); 18 of 22 defined excluded, bootstrap 20.2 s (was ~25 min). Full batch 62.86 s, 13 passed / 18 skipped (known QVAC-18460 iOS skips) / 0 failed.bootstrap()called without args:allowedDepsstaysundefined, manager warms everything as before.CI
Two full
QVAC Tests (sdk)runs against this PR (using the matching preview build oftetherto/qvac-test-suite#70), both fully green:--filter translation(8 deps)--filter registry(0 deps)keeping 8 dep(s); 16 of 24 defined excludedkeeping 0 dep(s); 24 of 24 defined excludedBootstrap completed inMobile, same two runs:
run-producer: 22m12s → 2m31s (test set 52 → 11).0/52 → 1/52 completedtransition shows bootstrap ~18.5 min for 8 deps on S25 Ultra (down from ~30 min full-registry warm seen in run 25672716968). For the 0-dep run the device starts the first test ~10 s after register.run-producer: 6m40s → 3m04s, noNo consumers connectedtimeouts. 49 pass / 3 skip vs 11 / 0.📌 Follow-ups before merge
@tetherto/qvac-test-suiteis currently pinned to0.6.3-tmp.pr-70.runid-25717564206(preview build oftetherto/qvac-test-suite#70). Once that PR lands and0.6.3is published, this dep moves to^0.6.3.