Add Jarvis mode: Sonnet 4.6 + double-clap status briefing#5
Conversation
- Switch all three Claude sessions (main, floating, observer) from claude-opus-4-6 to claude-sonnet-4-6 for cost efficiency - Add ClapDetector.swift: background mic listener that detects two sharp transients (RMS > 0.12 from quiet < 0.03) within 600ms - Add ChatPrompts.statusBriefing: concise spoken-style status prompt that instructs Claude to use speak_response tool automatically - Add ChatProvider.sendStatusBriefing(): sends the briefing query to the floating session with the status prompt as a suffix - Wire ClapDetector into PushToTalkManager.setup() so it starts on app launch; on double-clap fires sendStatusBriefing() when idle Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
m13v
left a comment
There was a problem hiding this comment.
Hey @arinunezm, thanks for the contribution! Cool concept. A few notes:
Model switch (Opus → Sonnet): We actually already made this change across the codebase (Apr 5-13) in ShortcutSettings, acp-bridge, ChatProvider, and DetachedChatWindow. So this part of the PR is redundant and would conflict with current main.
Clap detector feedback:
The idea is fun, but the current implementation has some significant reliability concerns:
-
No spectral analysis. The detector uses pure RMS energy detection, which means it can't distinguish a clap from typing, coughing, a door closing, snare hits in music, or plosive consonants in speech. Claps have a specific broadband frequency signature that could be detected with even a basic FFT, but raw energy alone will produce a lot of false positives in real environments.
-
Hardcoded thresholds. The 0.12 clap / 0.03 quiet thresholds are fixed with no calibration. Different microphones have different gain levels, and a quiet home office vs. a coffee shop will have completely different RMS baselines. There's no adaptive noise floor or startup calibration.
-
Always-on by default. The detector starts listening on app launch with no user opt-in. For a feature that uses the microphone continuously, users should explicitly enable it.
-
Environment dependency. Without adaptive thresholding, this will work great in one specific room and fail everywhere else.
What would help move this forward:
Before we can evaluate merging this, it'd be great to hear more from you on:
- What are the concrete use cases you're solving? When would someone want a status briefing via double-clap vs. just asking Fazm directly?
- How do you envision handling the false positive problem in practice?
- Would you be open to making this an opt-in feature with a calibration step?
The core infrastructure (mic access, PTT integration) is solid, but the detection algorithm needs more work to be production-ready. Happy to discuss further!
Summary
ChatProvider.sendStatusBriefing()→ Claude (floating session) generates a concise spoken-style status update →speak_responsetool reads it aloud via DeepGram TTSHow it works
Detection algorithm
Test plan
./run.sh— confirm logs showclaude-sonnet-4-6at warmupClapDetector: background mic listening startedin/private/tmp/fazm-dev.logDOUBLE CLAP detected!in logs🤖 Generated with Claude Code