Skip to content

refactor: move engines to engines/{parakeet,tts} (QIP PR 1)#95

Merged
GustavoA1604 merged 14 commits into
masterfrom
feat/engines-move
Jul 21, 2026
Merged

refactor: move engines to engines/{parakeet,tts} (QIP PR 1)#95
GustavoA1604 merged 14 commits into
masterfrom
feat/engines-move

Conversation

@ogad-tether

Copy link
Copy Markdown

What

PR 1 of the repo-reorg QIP (#94) — the symmetric engines/ layout:

  • parakeet-cpp/engines/parakeet/, tts-cpp/engines/tts/ as a rename-only commit (318 files, all recorded R100 — git log --follow/blame survive), followed by a separate commit updating the three engine workflows' path filters and source dirs.
  • No content changes to any engine source file.

Based on feat/engines-ci (#93) so the diff here is only the move; retarget to master after #93 merges.

Validation

  • Both engines configure cleanly from the new paths against system ggml (local, macOS arm64).
  • Full CI matrix runs on this PR across the moved paths: linux + macos build+test, windows build+test, android/ios compile smoke.

Coordination

The registry's parakeet-cpp / tts-cpp vcpkg ports reference the old paths — the next pin bump after this merges must carry the port-side path updates (normal 3-PR propagation; flagged in QIP #94's migration plan).

🤖 Generated with Claude Code

ogad-tether and others added 10 commits July 17, 2026 09:59
… QIP (PR 0)

Nothing in this repo's CI built or tested our engines until now — real
verification happened three repos downstream in tetherto/qvac. This adds
path-filtered workflows that, per PR:

- build qvac-ext-ggml@speech (CPU-only, cached by branch-tip SHA — the
  same fork the ggml-speech vcpkg port ships)
- build each engine + all test harnesses against it as system ggml
- run the non-GPU ctest lanes (`-LE 'gpu|perf'`); model-dependent tests
  auto-DISABLE to "Not Run" per the existing harness design

Validated locally on macOS arm64: tts lane 68/68 pass, parakeet lane
green (1 runnable test model-free; parity suites need staged GGUF
fixtures — follow-up documented in the QIP). GPU lanes are stubbed
behind workflow_dispatch until self-hosted GPU runners exist.

docs/QIP-speech-repo-reorg.md is the reorg proposal this is "PR 0" of
(symmetric engines/ layout, upstream vendored as a minimally-divergent
git subtree under third_party/). The workflows land against today's
paths; the reorg PRs only touch the path filters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… QIP to #94

First run of the new tts CI lane caught a real cross-platform bug:
test-campplus and test-campplus-backward-parity compile src/campplus.cpp
(which calls tts_cpp::detail::init_cpu_backend()) without
src/backend_selection.cpp, where that symbol lives. Apple's toolchain
happened to optimize the reference away; GNU ld on Linux fails with an
undefined reference — and Linux-with-tests was never built by any CI
until now. Mirror the source list test-voice-embedding already uses.

Also drop docs/QIP-speech-repo-reorg.md from this branch — the QIP now
has its own approval PR (#94); this PR is CI + fixes only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ads)

On ubuntu the test binaries resolve libqvac-speech-ggml.so via their
build RUNPATH, but RUNPATH is not consulted for that library's own
transitive dependencies (libqvac-speech-ggml-cpu.so.0 et al), so 35 tts
tests failed at exec with "cannot open shared object file". macOS is
unaffected (@rpath resolution walks the executable's rpath stack for
the whole load chain). Point LD_LIBRARY_PATH at the ggml install dir
for the ctest step in both lanes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e engines

engines-cross-ci.yml: engine x platform matrix complementing the
linux/mac lanes —
- windows-2022: MSVC, static system ggml, full build + non-GPU ctest
- android: NDK arm64-v8a compile smoke (no device to run tests on)
- ios: device arm64 compile smoke, Metal embedded (flags mirror
  build-xcframework.sh)

iOS recipe validated locally (ggml + both engines compile clean against
the iphoneos 26.2 SDK). On-device e2e remains downstream in
tetherto/qvac; these lanes front-load cross-compile/link breaks ahead
of the repo-reorg moves (QIP #94).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ts (rename-only)

Pure `git mv` with zero content changes so Git records 100% renames and
`git log --follow` / blame survive. Establishes the symmetric engines/
layout from the repo-reorg QIP (#94, PR 1); path fixups follow in the
next commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Path-filter and source-dir fixups for the rename-only move in the
previous commit. No workflow logic changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…amples in cross lanes

The new Windows lane's first run caught 9 test files calling POSIX
setenv/unsetenv, which MSVC's CRT doesn't provide — add
test/test_env_portable.h (same-named _putenv_s shims on _MSC_VER,
plain <cstdlib> elsewhere) and include it where used.

Also -DPARAKEET_BUILD_EXAMPLES=OFF in the windows/android/ios lanes:
the live-mic examples hand GCC-style warning flags to cl.exe and use
mic-capture APIs that don't exist on the iOS device SDK; examples are
dev conveniences, not part of the cross-platform surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/engines-cross-ci.yml
#	engines/tts/test/test_env_portable.h
test-gguf-stream and test-lavasr-enhancer-ggml consulted only TMPDIR
with a hard "/tmp" fallback — Windows sets TEMP/TMP and has no /tmp, so
both failed on the new Windows lane. Add test_tmpdir() to
test_env_portable.h (TMPDIR -> TEMP -> TMP -> "/tmp") and use it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ogad-tether and others added 2 commits July 17, 2026 16:21
…env shim, GPU flags in gpu stubs

- All ggml provisioning steps now fetch the exact SHA the cache key was
  derived from (git init + fetch --depth 1 <sha> + checkout FETCH_HEAD)
  instead of cloning the branch tip with a swallowed checkout — the tip
  advancing between ls-remote and clone could poison the cache with a
  commit the key never described. Applies to parakeet-ci, tts-ci and
  all three engines-cross-ci jobs.
- test_env_portable.h: the MSVC setenv shim now honors overwrite=0
  (existing variable left untouched, return 0) instead of silently
  diverging from POSIX.
- GPU stub jobs: build ggml with -DGGML_VULKAN=ON (the backend the
  -L gpu suites target; Metal is default-on on Apple hosts) so a
  dispatch run can't falsely pass against a CPU-only ggml; flag set to
  be revisited when the runner fleet exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GustavoA1604
GustavoA1604 previously approved these changes Jul 17, 2026
@ogad-tether ogad-tether self-assigned this Jul 21, 2026
mexxik
mexxik previously approved these changes Jul 21, 2026
@ogad-tether
ogad-tether changed the base branch from feat/engines-ci to master July 21, 2026 09:38
@ogad-tether
ogad-tether dismissed stale reviews from mexxik and GustavoA1604 July 21, 2026 09:38

The base branch was changed.

# Conflicts:
#	.github/workflows/engines-cross-ci.yml
#	.github/workflows/parakeet-ci.yml
#	.github/workflows/tts-ci.yml
#	engines/tts/test/parler/test_bpe_tokenizer.cpp
#	engines/tts/test/parler/test_dac.cpp
#	engines/tts/test/parler/test_decoder.cpp
#	engines/tts/test/parler/test_delay.cpp
#	engines/tts/test/parler/test_description.cpp
#	engines/tts/test/parler/test_e2e.cpp
#	engines/tts/test/parler/test_engine.cpp
#	engines/tts/test/parler/test_sched_equivalence.cpp
#	engines/tts/test/parler/test_t5.cpp
#	engines/tts/test/parler/test_text_norm.cpp
#	engines/tts/test/parler/test_tokenizer.cpp
#	engines/tts/test/test_env_portable.h
@github-actions

Copy link
Copy Markdown

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

@ogad-tether

Copy link
Copy Markdown
Author

Resolved the master merge: retargeted to master post-#93; the conflicts were (a) the three workflow files (kept the engines/ paths — master's squash content is already contained in this branch) and (b) PR #92's new Parler-TTS engine, which landed on master under tts-cpp/ after this branch was cut — all 36 Parler files are relocated to engines/tts/ (git's directory-rename detection moved the tests; src/include/scripts moved explicitly). Validated locally at the new location: engines/tts standalone build clean, 70/70 non-GPU tests (incl. the four runnable Parler tests); umbrella (on the stacked #96) 75/75. CI re-running.

…hed test

First lane run against master's Parler merge caught two issues:

- 19 tts tests SIGILL'd on ubuntu: the lanes built ggml with the
  default GGML_NATIVE=ON (-march=native) and cached the install keyed
  only by ggml SHA + OS — hosted runner fleets mix CPU generations, so
  a cache built on an AVX-512 machine crashes on one without. Build
  ggml with GGML_NATIVE=OFF in every hosted lane (gpu stubs on
  self-hosted machines keep native) and bust the possibly-poisoned
  caches with a -portable key prefix.
- test/parler/test_sched_equivalence.cpp (new in #92, never built on
  Windows) uses POSIX setenv/unsetenv — include the existing
  test_env_portable.h shim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@GustavoA1604
GustavoA1604 merged commit 07e0983 into master Jul 21, 2026
64 of 91 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants