|
| 1 | +--- |
| 2 | +type: memory |
| 3 | +title: ADR-KAI4 THE LATENT VOICE: GNA ear P0 shipped (G-VOICE-0 8/8), Audex teardown, text-bypass roadmap |
| 4 | +description: Sovereign voice: KAI-3 GNA ear (silicon-gated) wired to mic + /v1/voice + console -> inject_frames -> 12B (G-VOICE-0 parity 8/8, live e2e proven). Audex: steal unified-input + 2-stage output split, reject discrete-token output. P0=plumbing GREEN; P1 vocab scale-up makes free speech legible; P3 voice head = text bypass. E=3840 not 1024; GNA is frozen INT16 conv-only leverage. |
| 5 | +timestamp: 2026-07-10T15:26:18Z |
| 6 | +resource: kairos e180cea; harness 1dedacb; engine 98f8313 |
| 7 | +sp_status: GREEN |
| 8 | +sp_gate: G-VOICE-0 parity 8/8 + live wav->ear->inject e2e |
| 9 | +sp_commit: kairos e180cea; harness 1dedacb; engine 98f8313 |
| 10 | +sp_repro: serve.bat agent; python harness_tests/g_voice0_parity.py; console voice button |
| 11 | +mem_kind: agent |
| 12 | +mem_addr: fdebbd495a530e25 |
| 13 | +mem_verified: unverified |
| 14 | +mem_lifecycle: active |
| 15 | +tags: [KAI-4, latent voice, GNA ear, inject_frames, audio placeholder 258881, Audex teardown, discrete token reject, voice head, GAN vocoder, V_sub vocab, G-VOICE-0, voice channel GUI, voxtral reuse, E=3840, WSL no GNA MMIO, agent, tier-2] |
| 16 | +mem_tier: full |
| 17 | +--- |
| 18 | + |
| 19 | +ADR-KAI4 THE LATENT VOICE (2026-07-11, operator voice campaign). GOAL: sovereign realtime bidirectional voice - straight into the 12B's residual audio channel, text bypassed where possible, offloaded to the deprecated Intel GNA 2.0 (frozen target = leverage). |
| 20 | +INVENTORY (do not re-derive): the INPUT half is BUILT + GATED ON SILICON. KAI-3 ear = log-mel(64, 40ms/640 hop, 16k) -> Conv1d*3(256) CTC encoder -> POT i16 GNA-legal (pad=0, head padded to 36ch/÷4) -> CTC-greedy -> softmax(tau=0.2)*W_sub(√H-scaled embed rows) -> [k×E=3840] on-manifold frames -> daemon inject_frames + audio placeholder 258881 via gemma4_kv_inject_seq. Gated: 7/8 pivot, i16==FP32 0.877, GNA_HW bit-parity (G-KAIROS-3-GNA-HW). ckpt C:\Users\Knack\audio_ctc.pt. Reuse: OUR voxtral-mini-realtime-rs (hound/rubato/rustfft/ring_buffer + L0 iGPU + flow-matching TTS). ★WSL2 has NO GNA MMIO - live pipeline is Windows-native. |
| 21 | +AUDEX-2B TEARDOWN (arXiv 2607.05196, actually a 30B-A3B MoE + a 2B; NVIDIA one-way NONCOMMERCIAL license so we don't ship their weights): STEAL = unified continuous-input-projection (validates our inject_frames seam, we gated it first) + the 2-STAGE OUTPUT SPLIT (even NVIDIA doesn't emit waveforms from the LLM: compact acoustic stream -> small causal/streaming decoder = exactly our Voice-Head->GAN) + <sound> placeholder/template discipline. REJECT = discrete audio tokens through the text softmax (vocab extension + RVQ + token-rate decode on the 2060 = the tokenization tax we exist to avoid; our output leaves as a CONTINUOUS conditioning vector, no backbone surgery) + their 157B-token encoder economics + GPU-resident audio I/O. |
| 22 | +GEMINI CORRECTIONS: manifold is E=3840 NOT 1024; ear is already trained+on-silicon (work is STREAMING it not building it); we serve the 12B not a 9B; GNA is inference-only INT16 conv/affine no-transpose-conv no-padding ch÷4 (vocoder must pass a POT gate like the ear, don't assume); iGPU/L0 is a 4th compute tier Gemini missed. Compute map: GNA=always-on ear+wake, iGPU=heavier audio nets, CPU=vocoder v0, 2060=100% Gemma. |
| 23 | +P0 SHIPPED (kairos e180cea): harness/voice (dsp.py VERBATIM from gen_audio_frames.py trainer; ear.py OV GNA->CPU ladder; service.py); gateway /v1/voice + /v1/voice/status; console voice channel (mic getUserMedia 16k -> client VAD segmentation -> PCM16 b64 -> /v1/voice -> GNA ear -> inject_frames -> streamed reply -> v0 speechSynthesis TTS; push-to-talk/auto/interrupt-on-speak-while-speaking); tools/voice_export_wsub.py (torch-free W_sub from safetensors bf16<<16). G-VOICE-0 PARITY 8/8 (live ear == gated KAI2 packets, cos 1.0000, device CPU - GNA_HW needs the driver live in this process). LIVE E2E proven: 6.72s wav -> ear 4 frames -> daemon inject (S1 prompt injected). HONEST P0 boundary: reply empty on arbitrary speech - the ear's legible vocab is only V_sub=32 (KAI-3 action/no-op corpus); P0 gates PLUMBING, P1 = vocab scale-up 32->512 (multi-voice TTS render via our crate) + retrain + GNA re-gate. Ladder: P1 ear grows + wake head; P2 our CPU-GAN vocoder replaces browser TTS; P3 VOICE HEAD (hidden->conditioning, trained on self-generated pairs) = text-bypass leap; P4 GNA vocoder port. Gates G-VOICE-0..LIVE in the ADR. |
0 commit comments