Skip to content

Add playback callback#445

Open
shura-v wants to merge 1 commit intoargmaxinc:mainfrom
shura-v:add-playback-callback
Open

Add playback callback#445
shura-v wants to merge 1 commit intoargmaxinc:mainfrom
shura-v:add-playback-callback

Conversation

@shura-v
Copy link
Copy Markdown
Contributor

@shura-v shura-v commented Mar 22, 2026

TTSKit currently exposes generation-time audio via SpeechCallback, which is great for streaming UX but not sufficient for features that need to stay synchronized with actual playback.

For example, playback-reactive visualizers or lip sync can drift because audio chunks may be generated and queued ahead of real output.

It would be useful to expose an optional playback-aligned callback from AudioOutput that fires from the actual playback path and provides the chunk being played, rather than the chunk being generated or enqueued.

As a bonus, the test path can now suppress audible playback, which makes playback-callback integration tests much less surprising.

@shura-v shura-v marked this pull request as draft March 22, 2026 03:22
@shura-v shura-v force-pushed the add-playback-callback branch from ddcfe7d to e2d336b Compare March 22, 2026 03:26
@shura-v shura-v marked this pull request as ready for review March 24, 2026 07:44
@ZachNagengast
Copy link
Copy Markdown
Contributor

@shura-v Thanks for the PR, this looks useful just want to make sure I understand the usage - this is intended to tap into what is actually being scheduled to go out to the playerNode? Makes sense to be able to get these samples as they are played rather than as they are generated, if that is the case - but it also sounds like with the suppression this would allow you to manage the playback with your own audio system, is that intended with this?

@shura-v
Copy link
Copy Markdown
Contributor Author

shura-v commented Apr 1, 2026

@ZachNagengast The main goal is playback-synchronized lip sync.

SpeechCallback reflects generation time, which can drift from audible playback if audio is produced ahead of output. This callback is intended to reflect the actual playback path so consumers can stay aligned with what was really played.

setOutputSuppressed(true) was added mainly for tests. Once the callback moved onto the real playback path, the integration test started producing audible sound, so suppression lets us exercise that path without actually hearing it.

I would not frame this as a first-class custom audio backend API. The intent here is playback-aligned synchronization, with silent test coverage as a practical addition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants