Skip to content

Add Flux to the .NET SDK: conversational STT (/v2/listen) and TTS (/v2/speak) — preview#417

Open
dg-coreylweathers wants to merge 11 commits into
mainfrom
feat/flux
Open

Add Flux to the .NET SDK: conversational STT (/v2/listen) and TTS (/v2/speak) — preview#417
dg-coreylweathers wants to merge 11 commits into
mainfrom
feat/flux

Conversation

@dg-coreylweathers

@dg-coreylweathers dg-coreylweathers commented Jul 14, 2026

Copy link
Copy Markdown

Flux for the .NET SDK — conversational STT (/v2/listen) and TTS (/v2/speak), preview

Adds Deepgram Flux to the .NET SDK as a marked preview, additively. This branch carries the full
Flux feature:

  • Flux STT (/v2/listen) — conversational speech-to-text with contextual turn detection
    (TurnInfo/turn lifecycle). (Previously reviewed; included here because it is not yet on main.)
  • Flux TTS (/v2/speak) — text-to-speech on both transports: streaming WebSocket (stream text in,
    synthesized audio out, turn by turn) and batch REST (single-request synthesis + callback mode). New in
    this PR.
  • A mechanical namespace rename dropping the redundant v2 segment from the Flux namespaces
    (Deepgram.Clients.Flux.v2.WebSocket → Deepgram.Clients.Flux.WebSocket), so Clients.Speak.* = Aura
    (v1 era) and Clients.Flux.* = Flux (v2 era). No behavior change.

Built against the deepgram-docs spec PR #990 (/v2/speak streaming + batch) and cross-checked against the
Python/JS/Java Fern SDKs.

Public surface (TTS)

  • ClientFactory.CreateFluxSpeakWebSocketClient() → IFluxSpeakWebSocketClient
    (Connect, SendText/SendFlush/SendClose, Stop, Subscribe(...) events for
    Audio/Connected/SpeechStarted/SpeechMetadata/Flushed/SessionMetadata/Warning/Error/Open/Close/Unhandled).
  • ClientFactory.CreateFluxSpeakRESTClient() → IFluxSpeakRESTClient
    (ToStream/ToFile for audio out, StreamCallBack for the {request_id} ack).
  • New models under Deepgram.Models.Flux.Speak.{WebSocket,REST}; interfaces in the shared Interfaces.v2.

Correctness notes baked in (and tested)

  • Streaming close is {"type":"Close"} — not the listen client's CloseStream (guarded by a test).
  • Binary audio is delivered as raw bytes and never reaches the JSON parser (base routes on
    WebSocketMessageType).
  • sample_rate/bit_rate are int? and serialize with no decimal (verified under a comma-decimal culture).
  • Unknown {"type":"FutureMessage"} → Unhandled, never throws, never routes to Error (forward-compat).
  • No speed/Interrupt/Configure (GA-only) or KeepAlive/Finalize (v1-only) on the surface (reflection test).

Validation

  • Full suite: 274 passed, 0 failed, 1 env-gated live skip (211 pre-existing unchanged; +63 Flux TTS).
  • Live staging run (PASS): streaming flux-alexis-en Speak→Flush→Close produced interleaved binary
    audio (288,000 bytes = 6000ms @ 24kHz, matching SpeechMetadata) → Flushed → SpeechMetadata →
    SessionMetadata → one clean Close. Batch validated for mp3, linear16 @ sample_rate=44100, and
    callback-ack. Verbatim captured frames are replayed offline in FluxSpeakParityTests.
  • Additive only: existing public surface unchanged; the STT namespace rename is mechanical and left all
    STT tests green.

Notes for the reviewer

  • Preview: (PREVIEW) on every public type + README section; nothing published.
  • One flag (non-blocking, not yet in prod): on production /v2/speak today the API-key Token
    scheme is rejected (400 "Authentication header is not a supported type") while staging accepts it. The
    SDK sends Authorization: token <key> for API keys; worth reconciling before GA (may want a Bearer
    access token in prod, or it's pre-EA gating).
  • Follow-ups (documented): GA Interrupt/Configure/ResetContext are stubbed as TODOs; an
    independent Python-SDK golden for the TTS parity test; and consider a larger/configurable close-grace so
    a Stop() mid-turn doesn't truncate a long turn's audio (the example waits for SpeechMetadata first).

🤖 Generated with Claude Code

dg-coreylweathers and others added 11 commits July 10, 2026 10:08
@dg-coreylweathers @claude
feat(flux): add Flux (v2 listen) WebSocket client as preview
aa6f4b0 
Adds conversational speech-to-text with contextual turn detection via the
Flux /v2/listen endpoint: dedicated Flux event/schema types (TurnInfo with
turn lifecycle events and word-level timestamps, Connected,
ConfigureSuccess/Failure, fatal Error), mid-stream Configure support, and
CloseStream-based shutdown that waits for final results. Flux sends no
KeepAlive/Finalize (v1-only messages the v2 endpoint rejects).

Additive only: classic listen path is unchanged. Includes offline unit
tests, Python-SDK-golden parity test harness, an env-gated live test, and
SDK + raw-WebSocket examples.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dg-coreylweathers @claude
test(flux): wire fixtures, Python-golden parity, live e2e; coordinate…
d7421b1 
…d Stop

Captures real /v2/listen frames (105, full turn lifecycle incl. EagerEndOfTurn
and EndOfTurn) as committed fixtures; golden expectations generated by the
Python SDK's typed models parsing the identical frames. Parity + live tests
now run: 211/211 with key, 211+1 skip without.

Flux client gains a coordinated Stop(): CloseStream is sent once, the client
waits for the server to flush final results and close, and a teardown racing
the receive loop no longer throws on the disposed socket. Empirical findings
encoded: turns end on detected silence/eot_timeout (not CloseStream), and
ConfigureSuccess carries an undocumented profanity_filter field (tolerated).

Adds a global test-env snapshot because existing auth tests clear
DEEPGRAM_API_KEY process-wide.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dg-coreylweathers @claude
refactor(flux): drop the v2 segment from Flux namespaces
2a460b5 
Rename the Flux (v2 listen / STT) client and model namespaces to drop the
redundant `v2` segment, which collided in meaning with the SDK-architecture
`v2` (e.g. Speak.v2 targets /v1/speak). "Flux" is a v2-API-only product, so
the segment disambiguated nothing.

  Deepgram.Clients.Flux.v2.WebSocket -> Deepgram.Clients.Flux.WebSocket
  Deepgram.Models.Flux.v2.WebSocket  -> Deepgram.Models.Flux.WebSocket

Interfaces (Interfaces.v2.IFluxWebSocketClient), the FluxWebSocketClient
wrapper, and ClientFactory.CreateFluxWebSocketClient() are unchanged. Purely
mechanical: no behavior change; full suite green (211 passed, 1 live skipped).

Coordinated with the incoming Flux TTS (/v2/speak) work, which lands under
Deepgram.Clients.Flux.Speak.*.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
feat(flux-tts): add Flux TTS (v2 speak) models, enum, and interfaces …
16a7a5b 
…(preview)

Preview types for Deepgram Flux text-to-speech (/v2/speak), both transports,
under Deepgram.Models.Flux.Speak.{WebSocket,REST} with IFluxSpeakWebSocketClient
and IFluxSpeakRESTClient in the shared Interfaces.v2 bucket.

WebSocket: SpeakSchema (query options), SpeakMessage/ControlMessage (outbound
Speak/Flush/Close), SpeakType enum, and response records Connected, SpeechStarted,
SpeechMetadata (+ nested ControlsApplied), Flushed, SessionMetadata, Warning,
Error, Audio (binary), plus Open/Close/Unhandled wrappers over Common.

REST: SpeakSchema (integer sample_rate/bit_rate), TextSource ({text} body),
SyncResponse (audio stream + metadata), AsyncResponse (callback ack).

Lenient parsing throughout (all inbound fields nullable). No Interrupt/Configure/
speed (GA-only). Additive; library builds clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
feat(flux-tts): add Flux TTS streaming WebSocket client (preview)
faf2eec 
Deepgram.Clients.Flux.Speak.WebSocket.Client : AbstractWebSocketClient,
IFluxSpeakWebSocketClient, plus the FluxSpeakWebSocketClient wrapper and
ClientFactory.CreateFluxSpeakWebSocketClient().

Mirrors the Flux STT client (reused WS base, Subscribe/EventHandler events,
grace-drain SendClose, coordinated Stop) with the direction inverted:
ProcessBinaryMessage raises an AudioResponse (synthesized audio out) instead
of ignoring binary. Outbound is SendText/SendFlush/SendClose ({"type":"Speak"|
"Flush"|"Close"} - note Close, not the STT CloseStream). ProcessTextMessage
dispatches Connected/SpeechStarted/SpeechMetadata/Flushed/SessionMetadata/
Warning/Error; unknown types route to Unhandled and never throw. URI strips the
default /v1 and targets v2/speak; query building repeats tag and emits integer
sample_rate with no decimal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
feat(flux-tts): add Flux TTS batch REST client (preview)
706aeb0 
Deepgram.Clients.Flux.Speak.REST.Client : Abstractions.v2.AbstractRestClient,
IFluxSpeakRESTClient, plus the FluxSpeakRESTClient wrapper and
ClientFactory.CreateFluxSpeakRESTClient().

Mirrors the Speak v1 REST client: ToStream/ToFile return the synthesized audio
(binary) with response-header metadata; StreamCallBack sets the callback and
returns the {request_id} ack - covering both response modes. GetUri strips the
default /v1 and targets /v2/speak; query params (integer sample_rate/bit_rate,
repeated tag, lowercase bool mip_opt_out) go through QueryParameterUtil.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
test(flux-tts): add offline unit, wire-shape, and dual-mode tests
dedbf49 
FluxSpeakClientTests (WebSocket, 43) and FluxSpeakRestClientTests (batch, 17):
- wire-shape: /v2/speak URL routing (default/custom/versioned base), every query
  param name, integer sample_rate/bit_rate with no decimal under de-DE, repeated
  tag, lowercase mip_opt_out
- outbound JSON: Speak/Flush/Close frames (asserts Close, not CloseStream)
- inbound parse: all 7 control types incl. nested controls_applied and the
  model_uuids array; lenient parsing of missing/extra fields
- binary/text discrimination: a binary frame surfaces raw bytes on the Audio
  event, never hits the JSON parser
- forward-compat: {"type":"FutureMessage"} -> Unhandled, no throw, no Error
- batch dual-response: ToStream rejects a callback, StreamCallBack requires
  exactly one, AsyncResponse ack parses
- EA guards: no KeepAlive/Finalize/Interrupt/Configure/speed on the surface
- auth: both API key and access token carried on both transports

Full suite: 271 passed (was 211), 1 live skip, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
docs(flux-tts): add streaming and batch TTS examples
8c3059b 
examples/text-to-speech/websocket/flux (FluxSpeak): connect with flux-alexis-en,
SendText -> SendFlush, collect interleaved binary audio to output.raw, observe
Flushed/SpeechMetadata/SessionMetadata, clean Stop().

examples/text-to-speech/rest/flux (FluxSpeakBatch): ToFile for mp3 and for
linear16 wav at sample_rate=44100, plus a commented callback-mode snippet.

Both show the base-address override for targeting staging and are registered in
Deepgram.Dev.sln. Both build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
docs(flux-tts): add Flux Text to Speech (Preview) README section
8a5ff89 
New "Flux Text to Speech (Preview)" section (TOC + body) covering both
transports: streaming WebSocket (SendText/SendFlush/Stop with the AudioResponse
event) and batch REST (ToFile + callback), marked preview and calling out that
the streaming close message is Close (not the listen client's CloseStream).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
test(flux-tts): cover Open/Close wire dispatch and subscribe wrapping
4ae7402 
Adds a test that an Open text frame is forwarded to Open subscribers and a wire
"Close" frame is not delivered as a Close event (real Close events come from
connection teardown), exercising the base-delegated Open/Close Subscribe
wrappers and the ProcessTextMessage Open/Close path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers @claude
test(flux-tts): add staging-captured frame replay test; fix example c…
3b85ba3 
…lose timing

Validated live against wss://api.staging.deepgram.com/v2/speak:
- Fixtures/FluxSpeak/frames.json: verbatim control frames captured from a real
  Speak->Flush->Close session (Connected with a plural model_uuids array,
  SpeechStarted, Flushed, SpeechMetadata with nested controls_applied,
  SessionMetadata), plus the audio byte-count (257280 bytes = 5.36s @ 24kHz,
  matching audio_duration_ms).
- FluxSpeakParityTests replays those frames through ProcessTextMessage and
  asserts the parsed values; CI-offline. (Independent Python-SDK golden remains
  a documented follow-up.)

Example fix: the streaming example now waits for SpeechMetadata (turn audio
complete) before Stop() instead of a fixed delay, so audio is not truncated by
closing mid-turn.

Full suite: 274 passed, 1 live skip, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dg-coreylweathers

dg-coreylweathers commented Jul 14, 2026

Copy link
Copy Markdown
Author

Closing — opened prematurely; retracting from origin. Work continues on the local branch.

@dg-coreylweathers dg-coreylweathers deleted the feat/flux branch July 14, 2026 13:53
@dg-coreylweathers dg-coreylweathers restored the feat/flux branch July 14, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GregHolmes GregHolmes Awaiting requested review from GregHolmes GregHolmes is a code owner
@deepgram-kiley deepgram-kiley Awaiting requested review from deepgram-kiley deepgram-kiley is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dg-coreylweathers