Skip to content

Support the legacy single-function SpeechDecoder and MultiCodeDecoder assets - #520

Merged
ZachNagengast merged 4 commits into
mainfrom
oss/speech-decoder-schema-detection
Aug 5, 2026
Merged

Support the legacy single-function SpeechDecoder and MultiCodeDecoder assets#520
ZachNagengast merged 4 commits into
mainfrom
oss/speech-decoder-schema-detection

Conversation

@EduardoPach

@EduardoPach EduardoPach commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

#494 and #513 moved the SpeechDecoder and the MultiCodeDecoder onto multifunction
assets, which dropped support for the legacy single-function W8A16 ones. This
restores both by detecting the layout at load time instead of assuming it.

SpeechDecoder

  • Probe MLModelAsset.functionNames; empty → load without a functionName.
    Requesting throughputOptimized against it errors instead of silently
    producing latency-mode audio.
  • Read the kv_cache_update_mask rank from the model (2 = legacy, 3 =
    multifunction). Not derivable from codesPerStep — both ranks exist at
    codesPerStep == 1, so SpeechDecoderCache takes it as a parameter.
  • ModelUtilities dimension reads are now bounds-safe; probing an
    unexpected-rank input used to trap.

MultiCodeDecoder

Same probe: the legacy asset is schema-identical to the multifunction stepped
graph, so it loads without a functionName and drives the same decode path.
.fused has no counterpart in a single-function asset, so asking for it reports
a configuration error naming the multifunction variant rather than quietly
running stepped. .stepped stays the default; both variants are supported.

Downloader

setupModels built a fresh TTSKitConfig for the download that dropped
versionDir and the six component variants. downloadPatterns is derived from
those, so any pinned variant was ignored — the downloader fetched the preset
defaults and the load then failed on the variant actually requested.

This is pre-existing (it reproduces on main), but it makes the legacy support
above unreachable through TTSKit(config:), so it is fixed here. It also means a
user who already has W8A16 and pins it currently pulls the ~1.1 GB
multifunction assets they don't need before failing to load; after the fix that
run downloads nothing.

Validation

Cache deleted and re-downloaded from scratch: all 40 files match their original
SHA-256. Same text/seed at --temperature 0, vs main:

comparison result
multifunction stepped / fused / throughputOptimized vs main bit-exact
legacy W8A16 vs multifunction stepped corr 0.99999, max abs 7.4e-3
pinned W8A16 against a cache that already has it 0 bytes downloaded

Both guards fire: legacy + fused and legacy + throughputOptimized each report
the single-function configuration error. 4 tests added for rank-2 geometry. The
macOS 14 path stays deleted — both layouts need the macOS 15 / iOS 18 MLTensor
floor.

One commit is unrelated to the decoders: it gates xcbeautify --renderer github-actions to a single matrix job, since eight jobs were annotating the same
warning eight times.

Support both the multifunction and legacy single-function assets by
probing the model's functions and update-mask rank instead of assuming
a layout.
Eight matrix jobs compile the same sources, so each compiler warning was
posted eight times on the PR diff.
@EduardoPach
EduardoPach requested review from ZachNagengast and a2they and removed request for a2they August 5, 2026 21:49
@EduardoPach

EduardoPach commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

This PR will allow people to do the following in case they have the previous
assets already downloaded and don't want to download the new
W8A16-multifunction ones:

import TTSKit

Task {
    let config = TTSKitConfig(
        multiCodeDecoderVariant: "W8A16",
        speechDecoderVariant: "W8A16"
    )
    let tts = try await TTSKit(config)
    let result = try await tts.generate(text: "Hello from TTSKit!")
    print("Generated \(result.audioDuration)s of audio at \(result.sampleRate)Hz")
}

Probe the asset's CoreML functions instead of always setting a
functionName, so the legacy single-function W8A16 asset loads and runs
the stepped path. Requesting fused against it now reports a clear
configuration error.
@EduardoPach EduardoPach changed the title Support the legacy single-function SpeechDecoder asset Support the legacy single-function SpeechDecoder and MultiCodeDecoder assets Aug 5, 2026
@EduardoPach EduardoPach changed the title Support the legacy single-function SpeechDecoder and MultiCodeDecoder assets Support the legacy single-function SpeechDecoder/MultiCodeDecoder asset Aug 5, 2026
setupModels built a fresh TTSKitConfig for the download that dropped
versionDir and the six component variants. downloadPatterns is derived
from those, so any pinned variant was ignored: the downloader fetched the
preset defaults and the load then failed on the variant actually asked
for.

Without this, the legacy assets restored here are unreachable through
TTSKit(config:) -- and a user who already has W8A16 and pins it pulls the
~1.1GB multifunction assets they do not need before failing to load.
@EduardoPach EduardoPach changed the title Support the legacy single-function SpeechDecoder/MultiCodeDecoder asset Support the legacy single-function SpeechDecoder and MultiCodeDecoder assets Aug 5, 2026
@ZachNagengast
ZachNagengast merged commit 7e830c5 into main Aug 5, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants