Skip to content

QVAC-18410 test[skiplog]: add e2e coverage for transcribe metadata flag (PR-1701)#1883

Merged
Victor-Rodzko merged 5 commits into
mainfrom
test/aqa-pr-1701-transcribe-metadata
May 13, 2026
Merged

QVAC-18410 test[skiplog]: add e2e coverage for transcribe metadata flag (PR-1701)#1883
Victor-Rodzko merged 5 commits into
mainfrom
test/aqa-pr-1701-transcribe-metadata

Conversation

@Victor-Rodzko
Copy link
Copy Markdown
Contributor

@Victor-Rodzko Victor-Rodzko commented May 4, 2026

Note: be concise and prefer bullet points.

🎯 What problem does this PR solve?

📝 How does it solve it?

  • New transcription tests in transcription-tests.ts:
    • transcription-metadata-batchtranscribe({ metadata: true }) returns TranscribeSegment[].
    • transcription-metadata-streaming — drives the live bidirectional transcribeStream({ modelId, metadata: true }) session: decodes a 16kHz mono WAV fixture to f32le, pumps it through session.write() in chunks plus a trailing silence pad, then iterates yielded segments. Exercises transcribeStreamDuplexMetadata / processLineMetadata — not the deprecated upfront-audio overload.
  • New sad-path in parakeet-tests.ts:
    • parakeet-tdt-metadata-rejectedmetadata: true is whisper-only; parakeet engine must throw. Pins errorContains: "does not support metadata" so unrelated failures (model load, audio I/O) can't masquerade as a successful rejection.
  • Shared helpers in tests/shared/transcription-segments.ts:
    • validateSegments() — asserts shape { text, startMs, endMs, append, id } per segment, plus non-decreasing startMs and id so emission stays in audio-time order (append: true revisions may repeat a startMs). Content-agnostic — transcript correctness is already covered by existing transcription tests.
    • runMetadataStreamDuplex() — shared driver for the live duplex metadata flow (WAV decode → f32le → chunked write()end() → collect → validate). Used by both desktop and mobile executors.
  • Desktop + mobile (Bare/RN) executor parity for all three flows.

🧪 How was it tested?

  • Desktop (macOS, Bare): npx qvac-test run:local:desktop --filter "transcription-metadata-|parakeet-tdt-metadata-rejected" → 3/3 ✅.
  • iOS (physical iPhone): same filter → 3/3 ✅.

@Victor-Rodzko Victor-Rodzko requested a review from a team as a code owner May 4, 2026 14:23
@Victor-Rodzko Victor-Rodzko marked this pull request as draft May 4, 2026 14:24
@Victor-Rodzko Victor-Rodzko marked this pull request as draft May 4, 2026 14:24
@Victor-Rodzko Victor-Rodzko force-pushed the test/aqa-pr-1701-transcribe-metadata branch from 865eca4 to af28af9 Compare May 4, 2026 14:28
@Victor-Rodzko Victor-Rodzko force-pushed the test/aqa-pr-1701-transcribe-metadata branch from af28af9 to 11058c1 Compare May 8, 2026 14:17
@Victor-Rodzko Victor-Rodzko force-pushed the test/aqa-pr-1701-transcribe-metadata branch from 11058c1 to bc0eec0 Compare May 11, 2026 15:06
@Victor-Rodzko Victor-Rodzko force-pushed the test/aqa-pr-1701-transcribe-metadata branch from 7cc16e6 to 70abedd Compare May 11, 2026 16:11
@Victor-Rodzko Victor-Rodzko marked this pull request as ready for review May 11, 2026 16:58
@Victor-Rodzko Victor-Rodzko requested a review from a team as a code owner May 11, 2026 16:58
NamelsKing
NamelsKing previously approved these changes May 12, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 12, 2026

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (1/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*

@opaninakuffo
Copy link
Copy Markdown
Contributor

clean PR. tight scope: covers the three new code paths (transcribe({ metadata: true }), transcribeStream({ metadata: true }) duplex, parakeet rejection), and the parakeet error pin ("does not support metadata") is anchored to a real, engine-specific substring in parakeet-transcription/plugin.ts so it can't accidentally pass on a different failure. validators stay content-agnostic — nice. desktop+mobile parity maintained.
one minor non blocking thing below.

Comment thread packages/sdk/tests-qvac/tests/shared/transcription-segments.ts
Victor-Rodzko and others added 3 commits May 13, 2026 10:51
…t metadata (PR-1701)

- transcription-metadata-batch / transcription-metadata-streaming:
  validate TranscribeSegment[] shape returned by Whisper when metadata: true
- parakeet-tdt-metadata-rejected: assert metadata flag is rejected by
  non-Whisper engines (Parakeet)
- shared validateSegments() helper, content-agnostic shape check
- desktop + mobile executor parity for transcription and parakeet flows

Also folds in a tooling fix surfaced while authoring these tests:
tests-qvac/package.json gains clean:sdk-snapshot which wipes
node_modules/@qvac/sdk plus the iOS/Android consumer build snapshots
before reinstalling, so install:build:full no longer reuses a stale
@qvac/sdk copy on any platform.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions
Copy link
Copy Markdown

QVAC E2E — ios — running

View run
Config: suite=smoke · filter=(none) · exclude=(none)

@github-actions
Copy link
Copy Markdown

QVAC E2E — windows — running

View run
Config: suite=smoke · filter=(none) · exclude=(none)

@github-actions
Copy link
Copy Markdown

QVAC E2E — android — running

View run
Config: suite=smoke · filter=(none) · exclude=(none)

@github-actions
Copy link
Copy Markdown

QVAC E2E — linux — running

View run
Config: suite=smoke · filter=(none) · exclude=(none)

@github-actions
Copy link
Copy Markdown

QVAC E2E — macos — running

View run
Config: suite=smoke · filter=(none) · exclude=(none)

@kinsta
Copy link
Copy Markdown

kinsta Bot commented May 13, 2026

Preview deployments for qvac-docs-staging ⚡️

Status Branch preview Commit preview
✅ Ready Visit preview Visit preview

Commit: dcbb92b24bd53ca0315dd7b903386362c9e51fcc

Deployment ID: 975834a5-724d-4204-9ef9-2cd79c9db814

Static site name: qvac-docs-staging-fazwv

@Victor-Rodzko
Copy link
Copy Markdown
Contributor Author

/review

@Victor-Rodzko Victor-Rodzko merged commit 764a23b into main May 13, 2026
23 checks passed
@Victor-Rodzko Victor-Rodzko deleted the test/aqa-pr-1701-transcribe-metadata branch May 13, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-e2e-smoke Triggers smoke e2e test suite [Currently SDK-only] tier1 verified Authorize secrets / label-gate in PR workflows verify

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants