Voice support? Anyone interested in connecting AI TTS/STT up to this?

[gitcnd]

I've created an MCP for this lib, which works amazingly... but I want to be able to phone my AI and talk to it while I'm driving (hands free) so I'm wondering if anyone has an idea of...
a) how hard this might be to work out how to do, and then do
b) if anyone's interested in collaborating to make this real?

[abdessattar23]

Hey Chris, same here trying to add voice support, let's collaborate

[neoand]

Hi @gitcnd, @abdessattar23 — joining this thread because I've been
reverse-engineering WhatsApp VoIP via whatsmeow for several months and
might be useful here.

What I've worked out so far (high-level)

The VoIP pipeline has three layers, all required:

1. Signaling — <call> IQ stanzas exchanging offer + accept
   protobufs. whatsmeow already parses these (events.CallOffer,
   events.CallAccept) but the offer construction for outbound calls
   needs custom protobuf assembly: Call message with media,
   relaytoken, enc payloads.
2. Relay (TURN) — Meta runs STUN/TURN endpoints (relay-*.facebook.com
   on standard 3478). After signaling, both peers Allocate + Bind
   against the relay to exchange transport addresses. HMAC auth on the
   TURN messages uses the raw base64 string of the relay credentials key
   (24-byte ASCII as the HMAC-SHA1 secret — non-obvious, worth a separate
   note).
3. Media (SRTP) — Opus audio over RTP/SRTP, keys derived via HKDF
   from the signaling exchange. RTP timestamps + sequence numbers
   conform to RFC 3711.

I have all three running end-to-end in Go (Mexico-based test numbers
ringing the recipient's phone for ~20s before "Reconectando"), but I
hit a current blocker on audio format that I'm still debugging — the
callee receives + decrypts our packets but the WhatsApp app doesn't
seem to render them as a live call. Working hypothesis: DTX framing or
codec parameter mismatch.

On the TTS/STT use case specifically

For "phone my AI and talk while driving", the natural shape is:

• Outbound call from your whatsmeow client to your number
• Pipe inbound Opus → STT (e.g. Whisper) → LLM → TTS (e.g. Piper/Coqui)
  → outbound Opus
• All within the same Go process — pion/opus or gopus for the codec,
  pion/webrtc patterns for SRTP if you want a battle-tested SRTP layer

The biggest practical issue isn't the audio plumbing — it's getting
WhatsApp itself to accept your protobuf as a "real call". If anyone has
seen the SRTP MAC tag computation working with a non-WABA sender, I'd
trade notes.

What I can share

I'm happy to share:

• Annotated protobuf captures of working signaling exchanges
• The TURN HMAC key derivation (non-obvious, took two sessions to solve)
• WASP relay token format
• The exact s_o / s_t salt structure for SRTP HKDF

DM-equivalent here works (reply to this thread). My broader notes live
in a private NeoGo repo — happy to extract relevant bits if anyone
wants to compare wire captures.

Worth coordinating: even a community-maintained doc on "how WhatsApp
VoIP is structured at the protocol level" would save the next person
months. Currently every implementor (Schirrmacher, the WA-Calls
projects on GitHub, etc) reinvents the same pieces.