Skip to content

docs: QIP — reorganize the repo into a symmetric multi-engine speech layout#94

Open
ogad-tether wants to merge 2 commits into
masterfrom
feat/speech-repo-reorg-qip
Open

docs: QIP — reorganize the repo into a symmetric multi-engine speech layout#94
ogad-tether wants to merge 2 commits into
masterfrom
feat/speech-repo-reorg-qip

Conversation

@ogad-tether

@ogad-tether ogad-tether commented Jul 17, 2026

Copy link
Copy Markdown

What

The reorg proposal (docs/QIP-speech-repo-reorg.md) — this PR is the approval vehicle: review the plan, not code. No source or build changes here.

Proposed end state:

CMakeLists.txt                 # umbrella superbuild (feature-gated); ours
third_party/whisper.cpp/       # upstream via git subtree, minimally divergent + PATCHES.md
engines/
  whisper/                     # thin adapter over third_party
  parakeet/                    # git mv from parakeet-cpp/
  tts/                         # git mv from tts-cpp/
  qwen-asr/                    # future

Key points, from a repo audit (all verified against the tree):

  • "Pristine, never hand-edited" vendoring is not possible: we carry ~374 lines of patches on upstream files vs v1.9.1 (QVAC-21623 QKV fusion + logits slice in src/whisper.cpp, ggml backend-reg/vulkan). Policy amended to upstream-tracking, minimally divergent: deltas declared in PATCHES.md, enforced by a CI divergence-guard job; git subtree pull then conflicts only on the ~3 files we touch instead of today's whole-tree squashed re-apply.
  • ggml divergence is banned in-repo: canonical ggml patches live in qvac-ext-ggml@speech (the ggml-speech port source); the umbrella build forces system ggml so the vendored ggml/ is never compiled.
  • Concrete subtree conversion recipe (git subtree add can't target an existing prefix), phased PR plan (CI first → renames → vendor conversion → sync runbook), registry-port coordination, and upstream's new parakeet (C API) gated off vs our engines/parakeet (C++).
  • CI/testing scaffold: per-engine build+test lanes (already implemented and validated in ci: per-engine CI lanes for parakeet-cpp and tts-cpp (QIP PR 0) #93, which lands independently of this approval), GPU lane stubs, port-smoke follow-up, and the path to enabling the parity suites (hosted GGUF fixtures).

Approvals needed

Speech team lead · infra/registry owner · downstream (qvac) owner — table stubbed at the bottom of the doc.

🤖 Generated with Claude Code

@ogad-tether @claude
docs: QIP — reorganize the repo into a symmetric multi-engine speech …
7744f48 
…layout

Proposal: engines/{whisper,parakeet,tts} + upstream whisper.cpp vendored
as a minimally-divergent git subtree under third_party/whisper.cpp, with
an umbrella feature-gated CMake superbuild and in-repo per-engine CI.

Key deviations from the original sketch, based on a repo audit:
- "pristine, never hand-edited" vendoring is not possible — we carry
  ~374 lines of patches on upstream files (QKV fusion, logits slice,
  ggml backend-reg/vulkan); policy amended to upstream-tracking +
  PATCHES.md manifest + CI divergence guard
- ggml divergence moves entirely to qvac-ext-ggml@speech (the
  ggml-speech port source); the umbrella build forces system ggml
- concrete subtree conversion recipe (git subtree add can't target an
  existing prefix), phased PR plan, and upstream-sync runbook

CI scaffold (PR 0, #93) lands independently of this approval.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ogad-tether
ogad-tether requested review from a team as code owners July 17, 2026 09:03
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

ogad-tether added a commit that referenced this pull request Jul 17, 2026
@ogad-tether @claude
fix(tts): link backend_selection.cpp into campplus test targets; move…
97e8ab0 
… QIP to #94

First run of the new tts CI lane caught a real cross-platform bug:
test-campplus and test-campplus-backward-parity compile src/campplus.cpp
(which calls tts_cpp::detail::init_cpu_backend()) without
src/backend_selection.cpp, where that symbol lives. Apple's toolchain
happened to optimize the reference away; GNU ld on Linux fails with an
undefined reference — and Linux-with-tests was never built by any CI
until now. Mirror the source list test-voice-embedding already uses.

Also drop docs/QIP-speech-repo-reorg.md from this branch — the QIP now
has its own approval PR (#94); this PR is CI + fixes only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ogad-tether ogad-tether mentioned this pull request Jul 17, 2026
Merged
@ogad-tether ogad-tether self-assigned this Jul 17, 2026
ogad-tether added a commit that referenced this pull request Jul 17, 2026
@ogad-tether @claude
ci: add Windows build+test and Android/iOS compile-smoke lanes for th…
176336f 
…e engines

engines-cross-ci.yml: engine x platform matrix complementing the
linux/mac lanes —
- windows-2022: MSVC, static system ggml, full build + non-GPU ctest
- android: NDK arm64-v8a compile smoke (no device to run tests on)
- ios: device arm64 compile smoke, Metal embedded (flags mirror
  build-xcframework.sh)

iOS recipe validated locally (ggml + both engines compile clean against
the iphoneos 26.2 SDK). On-device e2e remains downstream in
tetherto/qvac; these lanes front-load cross-compile/link breaks ahead
of the repo-reorg moves (QIP #94).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 17, 2026
Merged
Merged
Merged
Draft
@ogad-tether @claude
docs(qip): address review feedback
96b62f9 
- Downstream validation policy: in-repo ctests are not sufficient to
  approve engine changes — a qvac overlay-port draft PR exercising all
  platforms/addons (desktop + Device Farm mobile e2e) is a required
  gate before the registry pin bump. Documents the exact recipe
  (overlay portfiles + vcpkg-configuration, label mechanics) and
  references the executed example (qvac#3310). Replaces the weaker
  port-smoke idea.
- PATCHES.md policy: manifest rows must be self-contained (what + why),
  no reliance on tracker tickets.
- CI table: reflect the shipped Windows build+test and Android/iOS
  compile-smoke lanes (engines-cross-ci.yml, green in PR 0).
- PR 2 deliverables: full root README (engines/models, install + run,
  cross-platform perf) — delivered on the PR 2 branch.
- Fix stale `ctest -L cpu` reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ogad-tether added a commit that referenced this pull request Jul 21, 2026
@ogad-tether @claude
ci: per-engine CI lanes for parakeet-cpp and tts-cpp (QIP PR 0) (#93)
0301210 
* ci: add per-engine CI lanes for parakeet-cpp and tts-cpp + repo-reorg QIP (PR 0)

Nothing in this repo's CI built or tested our engines until now — real
verification happened three repos downstream in tetherto/qvac. This adds
path-filtered workflows that, per PR:

- build qvac-ext-ggml@speech (CPU-only, cached by branch-tip SHA — the
  same fork the ggml-speech vcpkg port ships)
- build each engine + all test harnesses against it as system ggml
- run the non-GPU ctest lanes (`-LE 'gpu|perf'`); model-dependent tests
  auto-DISABLE to "Not Run" per the existing harness design

Validated locally on macOS arm64: tts lane 68/68 pass, parakeet lane
green (1 runnable test model-free; parity suites need staged GGUF
fixtures — follow-up documented in the QIP). GPU lanes are stubbed
behind workflow_dispatch until self-hosted GPU runners exist.

docs/QIP-speech-repo-reorg.md is the reorg proposal this is "PR 0" of
(symmetric engines/ layout, upstream vendored as a minimally-divergent
git subtree under third_party/). The workflows land against today's
paths; the reorg PRs only touch the path filters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): link backend_selection.cpp into campplus test targets; move QIP to #94

First run of the new tts CI lane caught a real cross-platform bug:
test-campplus and test-campplus-backward-parity compile src/campplus.cpp
(which calls tts_cpp::detail::init_cpu_backend()) without
src/backend_selection.cpp, where that symbol lives. Apple's toolchain
happened to optimize the reference away; GNU ld on Linux fails with an
undefined reference — and Linux-with-tests was never built by any CI
until now. Mirror the source list test-voice-embedding already uses.

Also drop docs/QIP-speech-repo-reorg.md from this branch — the QIP now
has its own approval PR (#94); this PR is CI + fixes only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: export LD_LIBRARY_PATH for the ctest step (transitive ggml .so loads)

On ubuntu the test binaries resolve libqvac-speech-ggml.so via their
build RUNPATH, but RUNPATH is not consulted for that library's own
transitive dependencies (libqvac-speech-ggml-cpu.so.0 et al), so 35 tts
tests failed at exec with "cannot open shared object file". macOS is
unaffected (@rpath resolution walks the executable's rpath stack for
the whole load chain). Point LD_LIBRARY_PATH at the ggml install dir
for the ctest step in both lanes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: add Windows build+test and Android/iOS compile-smoke lanes for the engines

engines-cross-ci.yml: engine x platform matrix complementing the
linux/mac lanes —
- windows-2022: MSVC, static system ggml, full build + non-GPU ctest
- android: NDK arm64-v8a compile smoke (no device to run tests on)
- ios: device arm64 compile smoke, Metal embedded (flags mirror
  build-xcframework.sh)

iOS recipe validated locally (ggml + both engines compile clean against
the iphoneos 26.2 SDK). On-device e2e remains downstream in
tetherto/qvac; these lanes front-load cross-compile/link breaks ahead
of the repo-reorg moves (QIP #94).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): portable setenv/unsetenv shim for MSVC test builds; skip examples in cross lanes

The new Windows lane's first run caught 9 test files calling POSIX
setenv/unsetenv, which MSVC's CRT doesn't provide — add
test/test_env_portable.h (same-named _putenv_s shims on _MSC_VER,
plain <cstdlib> elsewhere) and include it where used.

Also -DPARAKEET_BUILD_EXAMPLES=OFF in the windows/android/ios lanes:
the live-mic examples hand GCC-style warning flags to cl.exe and use
mic-capture APIs that don't exist on the iOS device SDK; examples are
dev conveniences, not part of the cross-platform surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): portable temp-dir fallback for tests that write scratch GGUFs

test-gguf-stream and test-lavasr-enhancer-ggml consulted only TMPDIR
with a hard "/tmp" fallback — Windows sets TEMP/TMP and has no /tmp, so
both failed on the new Windows lane. Add test_tmpdir() to
test_env_portable.h (TMPDIR -> TEMP -> TMP -> "/tmp") and use it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci+fix: address #93 review — exact-SHA ggml fetch, POSIX-faithful setenv shim, GPU flags in gpu stubs

- All ggml provisioning steps now fetch the exact SHA the cache key was
  derived from (git init + fetch --depth 1 <sha> + checkout FETCH_HEAD)
  instead of cloning the branch tip with a swallowed checkout — the tip
  advancing between ls-remote and clone could poison the cache with a
  commit the key never described. Applies to parakeet-ci, tts-ci and
  all three engines-cross-ci jobs.
- test_env_portable.h: the MSVC setenv shim now honors overwrite=0
  (existing variable left untouched, return 0) instead of silently
  diverging from POSIX.
- GPU stub jobs: build ggml with -DGGML_VULKAN=ON (the backend the
  -L gpu suites target; Metal is default-on on Apple hosts) so a
  dispatch run can't falsely pass against a CPU-only ggml; flag set to
  be revisited when the runner fleet exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
GustavoA1604 pushed a commit that referenced this pull request Jul 21, 2026
@ogad-tether @claude
refactor: move engines to engines/{parakeet,tts} (QIP PR 1) (#95)
07e0983 
* ci: add per-engine CI lanes for parakeet-cpp and tts-cpp + repo-reorg QIP (PR 0)

Nothing in this repo's CI built or tested our engines until now — real
verification happened three repos downstream in tetherto/qvac. This adds
path-filtered workflows that, per PR:

- build qvac-ext-ggml@speech (CPU-only, cached by branch-tip SHA — the
  same fork the ggml-speech vcpkg port ships)
- build each engine + all test harnesses against it as system ggml
- run the non-GPU ctest lanes (`-LE 'gpu|perf'`); model-dependent tests
  auto-DISABLE to "Not Run" per the existing harness design

Validated locally on macOS arm64: tts lane 68/68 pass, parakeet lane
green (1 runnable test model-free; parity suites need staged GGUF
fixtures — follow-up documented in the QIP). GPU lanes are stubbed
behind workflow_dispatch until self-hosted GPU runners exist.

docs/QIP-speech-repo-reorg.md is the reorg proposal this is "PR 0" of
(symmetric engines/ layout, upstream vendored as a minimally-divergent
git subtree under third_party/). The workflows land against today's
paths; the reorg PRs only touch the path filters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): link backend_selection.cpp into campplus test targets; move QIP to #94

First run of the new tts CI lane caught a real cross-platform bug:
test-campplus and test-campplus-backward-parity compile src/campplus.cpp
(which calls tts_cpp::detail::init_cpu_backend()) without
src/backend_selection.cpp, where that symbol lives. Apple's toolchain
happened to optimize the reference away; GNU ld on Linux fails with an
undefined reference — and Linux-with-tests was never built by any CI
until now. Mirror the source list test-voice-embedding already uses.

Also drop docs/QIP-speech-repo-reorg.md from this branch — the QIP now
has its own approval PR (#94); this PR is CI + fixes only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: export LD_LIBRARY_PATH for the ctest step (transitive ggml .so loads)

On ubuntu the test binaries resolve libqvac-speech-ggml.so via their
build RUNPATH, but RUNPATH is not consulted for that library's own
transitive dependencies (libqvac-speech-ggml-cpu.so.0 et al), so 35 tts
tests failed at exec with "cannot open shared object file". macOS is
unaffected (@rpath resolution walks the executable's rpath stack for
the whole load chain). Point LD_LIBRARY_PATH at the ggml install dir
for the ctest step in both lanes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: add Windows build+test and Android/iOS compile-smoke lanes for the engines

engines-cross-ci.yml: engine x platform matrix complementing the
linux/mac lanes —
- windows-2022: MSVC, static system ggml, full build + non-GPU ctest
- android: NDK arm64-v8a compile smoke (no device to run tests on)
- ios: device arm64 compile smoke, Metal embedded (flags mirror
  build-xcframework.sh)

iOS recipe validated locally (ggml + both engines compile clean against
the iphoneos 26.2 SDK). On-device e2e remains downstream in
tetherto/qvac; these lanes front-load cross-compile/link breaks ahead
of the repo-reorg moves (QIP #94).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: move parakeet-cpp -> engines/parakeet, tts-cpp -> engines/tts (rename-only)

Pure `git mv` with zero content changes so Git records 100% renames and
`git log --follow` / blame survive. Establishes the symmetric engines/
layout from the repo-reorg QIP (#94, PR 1); path fixups follow in the
next commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: point workflows at engines/parakeet and engines/tts

Path-filter and source-dir fixups for the rename-only move in the
previous commit. No workflow logic changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): portable setenv/unsetenv shim for MSVC test builds; skip examples in cross lanes

The new Windows lane's first run caught 9 test files calling POSIX
setenv/unsetenv, which MSVC's CRT doesn't provide — add
test/test_env_portable.h (same-named _putenv_s shims on _MSC_VER,
plain <cstdlib> elsewhere) and include it where used.

Also -DPARAKEET_BUILD_EXAMPLES=OFF in the windows/android/ios lanes:
the live-mic examples hand GCC-style warning flags to cl.exe and use
mic-capture APIs that don't exist on the iOS device SDK; examples are
dev conveniences, not part of the cross-platform surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tts): portable temp-dir fallback for tests that write scratch GGUFs

test-gguf-stream and test-lavasr-enhancer-ggml consulted only TMPDIR
with a hard "/tmp" fallback — Windows sets TEMP/TMP and has no /tmp, so
both failed on the new Windows lane. Add test_tmpdir() to
test_env_portable.h (TMPDIR -> TEMP -> TMP -> "/tmp") and use it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci+fix: address #93 review — exact-SHA ggml fetch, POSIX-faithful setenv shim, GPU flags in gpu stubs

- All ggml provisioning steps now fetch the exact SHA the cache key was
  derived from (git init + fetch --depth 1 <sha> + checkout FETCH_HEAD)
  instead of cloning the branch tip with a swallowed checkout — the tip
  advancing between ls-remote and clone could poison the cache with a
  commit the key never described. Applies to parakeet-ci, tts-ci and
  all three engines-cross-ci jobs.
- test_env_portable.h: the MSVC setenv shim now honors overwrite=0
  (existing variable left untouched, return 0) instead of silently
  diverging from POSIX.
- GPU stub jobs: build ggml with -DGGML_VULKAN=ON (the backend the
  -L gpu suites target; Metal is default-on on Apple hosts) so a
  dispatch run can't falsely pass against a CPU-only ggml; flag set to
  be revisited when the runner fleet exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci+fix: portable ggml builds for hosted lanes; env shim for Parler sched test

First lane run against master's Parler merge caught two issues:

- 19 tts tests SIGILL'd on ubuntu: the lanes built ggml with the
  default GGML_NATIVE=ON (-march=native) and cached the install keyed
  only by ggml SHA + OS — hosted runner fleets mix CPU generations, so
  a cache built on an AVX-512 machine crashes on one without. Build
  ggml with GGML_NATIVE=OFF in every hosted lane (gpu stubs on
  self-hosted machines keep native) and bust the possibly-poisoned
  caches with a -portable key prefix.
- test/parler/test_sched_equivalence.cpp (new in #92, never built on
  Windows) uses POSIX setenv/unsetenv — include the existing
  test_env_portable.h shim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@ogad-tether ogad-tether

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ogad-tether