Skip to content

Commit a835eaf

Browse files
AzimovSclaude
andauthored
feat(detection): auto-detect meetings via microphone activity (macOS) (#35)
* feat(notifications): add MeetingDetected / MeetingEnded variants Scaffolds the notification plumbing for mic-activity meeting detection (landing in the next commit). Adds `MeetingDetected(String)` and `MeetingEnded(String)` enum variants alongside their settings toggles (`show_meeting_detected` / `show_meeting_ended`, default on), manager helpers, and the "Meeting detected — tap to start recording" / "Meeting ended — tap to stop recording" body text. Surfaces both in the About debug dropdown so they can be fired manually. * feat(detection): auto-detect meetings via microphone activity (macOS) Observes which non-Meetily apps hold the default input device. When a known meeting app (Zoom, Teams, FaceTime, Discord, Slack, any browser) sustains mic activity for 10s, fires a "Meeting detected — <app>" banner. Unknown apps wait 30s. During an active Meetily recording, a 30s silence window after mic release fires "Meeting ended — tap to stop recording". Matches the approach used by `char` (fastrepl/char) and Granola: no browser URL reading, no window title parsing, no new permissions. Implementation: - `state.rs`: pure state machine (`Idle → Sustaining → Detected → Ending → Idle`) parameterized over `Instant`; 23 unit tests cover priority upgrades, flicker, reacquire debouncing, dismissal cooldowns, and the "not recording" suppression path. - `matcher.rs`: hybrid allowlist + blocklist. Blocklist filters Meetily itself, Apple dictation/voice-memos, third-party whisper apps, and screen recorders. Allowlist maps bundle IDs to a cross- platform `App` enum (preserved for future Windows/Linux enablement). - `signals/mic_activity/macos.rs`: CoreAudio via `cidre`. Cheap gate on `kAudioDevicePropertyDeviceIsRunningSomewhere`, per-process enumeration only when hot. - `signals/mic_activity/stub.rs`: no-op fallback. Used on non-macOS targets so the detection service compiles and runs idle — no banners ever fire, but the task loop is harmless. - `service.rs`: 1s poll loop, `DetectionService` handle registered in Tauri state; shutdown gated on `RunEvent::Exit`. - `audio::recording_commands`: pushes recording state into the service via `try_state` so detection knows whether to fire meeting-ended banners without depending on audio internals. - Tauri command surface: `dismiss_detected_meeting`, `get_detection_state` for UI / agent observation. Events emitted as `meeting-detected` / `meeting-ended`. Scope: macOS only. Windows (WASAPI) and Linux (PulseAudio) samplers are deferred to follow-up PRs so they can be validated on real hardware and shipped behind a settings-level kill switch. The `App` enum and stub-sampler factory shape make adding those platforms a matter of slotting in platform files, not rearchitecting. * fix(detection): address code-review P1s + P2s (001, 002, 004, 005) - set_recording moved to Arc<AtomicBool> on DetectionService; poll loop syncs it into state before each advance. Removes the try_lock/spawn fallback that could reorder rapid start/stop calls and wrongly gate the MeetingEnded banner. - DetectorPhaseSnapshot.bundle_id exposed for non-idle phases so agents can act on get_detection_state alone (dismiss_detected_meeting requires the raw bundle_id). - meeting-detected / meeting-ended Tauri events gated on the matching notification preference and emit DetectedMeetingEvent { display_name } only. bundle_id stays Rust-side; agents read it via the command surface, not the event bus. - NSMicrophoneUsageDescription updated to cover the new process- enumeration capability introduced by CoreAudio mic-holder detection. The NotificationManager OnceCell migration (003) was dropped after review: the race it addressed is narrow (racing first-callers during startup eager-init) and the fallout benign (duplicate settings-file write, idempotent delegate re-registration). Not worth ~100 LOC of churn across every notification command handler. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 1e08fbb commit a835eaf

18 files changed

Lines changed: 1658 additions & 2 deletions

File tree

frontend/src-tauri/Info.plist

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
<plist version="1.0">
44
<dict>
55
<key>NSMicrophoneUsageDescription</key>
6-
<string>This application needs access to your microphone to record meeting audio.</string>
6+
<string>Meetily uses your microphone to record meetings you ask it to capture, and observes which apps are currently using the microphone so it can offer to auto-record when a meeting starts.</string>
77
<key>NSScreenCaptureUsageDescription</key>
88
<string>This application needs screen recording permission to capture system audio during meetings.</string>
99
<key>NSAudioCaptureUsageDescription</key>

frontend/src-tauri/src/audio/recording_commands.rs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,7 @@ pub async fn start_recording_with_meeting_name<R: Runtime>(
245245
// Set recording flag and reset speech detection flag
246246
info!("🔍 Setting IS_RECORDING to true and resetting SPEECH_DETECTED_EMITTED");
247247
IS_RECORDING.store(true, Ordering::SeqCst);
248+
notify_detection_recording_state(&app, true);
248249
reset_speech_detected_flag(); // Reset for new recording session
249250

250251
// Start optimized parallel transcription task and store handle
@@ -414,6 +415,7 @@ pub async fn start_recording_with_devices_and_meeting<R: Runtime>(
414415
// Set recording flag and reset speech detection flag
415416
info!("🔍 Setting IS_RECORDING to true and resetting SPEECH_DETECTED_EMITTED");
416417
IS_RECORDING.store(true, Ordering::SeqCst);
418+
notify_detection_recording_state(&app, true);
417419
reset_speech_detected_flag(); // Reset for new recording session
418420

419421
// Start optimized parallel transcription task and store handle
@@ -742,6 +744,7 @@ pub async fn stop_recording<R: Runtime>(
742744
// Set recording flag to false
743745
info!("🔍 Setting IS_RECORDING to false");
744746
IS_RECORDING.store(false, Ordering::SeqCst);
747+
notify_detection_recording_state(&app, false);
745748

746749
// Step 4.5: Prepare metadata for frontend (NO database save)
747750
// NOTE: We do NOT save to database here. The frontend will save after all transcripts are displayed.
@@ -794,6 +797,22 @@ pub async fn is_recording() -> bool {
794797
IS_RECORDING.load(Ordering::SeqCst)
795798
}
796799

800+
/// Sync form of `is_recording` for callers that aren't async — same
801+
/// atomic flag as the async version.
802+
pub fn is_recording_sync() -> bool {
803+
IS_RECORDING.load(Ordering::SeqCst)
804+
}
805+
806+
/// Push the current recording state into the detection service so it
807+
/// can gate MeetingEnded banners on whether the user is actually
808+
/// recording. No-op if the service isn't registered yet (rare — only
809+
/// during startup races or disabled-detection builds).
810+
fn notify_detection_recording_state<R: Runtime>(app: &AppHandle<R>, recording: bool) {
811+
if let Some(svc) = app.try_state::<crate::detection::service::DetectionService>() {
812+
svc.set_recording(recording);
813+
}
814+
}
815+
797816
/// Get recording statistics
798817
pub async fn get_transcription_status() -> TranscriptionStatus {
799818
TranscriptionStatus {
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
//! Tauri command surface for the detection module.
2+
//!
3+
//! Exposes the otherwise-private `DetectorState` API to the frontend
4+
//! and to agents via the invoke bridge.
5+
6+
use tauri::{State, Wry};
7+
8+
use crate::detection::service::DetectionService;
9+
use crate::detection::state::DetectorPhaseSnapshot;
10+
11+
/// Suppress detection events for this bundle for the dismissal cooldown
12+
/// (default 10 min). Used by the "ignore this app" UX path.
13+
#[tauri::command]
14+
pub async fn dismiss_detected_meeting(
15+
bundle_id: String,
16+
service: State<'_, DetectionService>,
17+
) -> Result<(), String> {
18+
service.dismiss(&bundle_id).await;
19+
Ok(())
20+
}
21+
22+
/// Return the detector's current phase, timing, and recording state.
23+
/// Useful for UI indicators + agent observation.
24+
#[tauri::command]
25+
pub async fn get_detection_state(
26+
service: State<'_, DetectionService>,
27+
) -> Result<DetectorPhaseSnapshot, String> {
28+
Ok(service.current_phase().await)
29+
}
30+
31+
// Keep Wry in scope so the generated handler types match the rest of
32+
// the app's invoke_handler registrations.
33+
#[allow(dead_code)]
34+
fn _check_handler_type(_: tauri::AppHandle<Wry>) {}
Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
//! Hybrid allowlist + blocklist for mic-activity based meeting detection.
2+
//!
3+
//! Decision order per candidate bundle:
4+
//! 1. Blocklist hit → filter out entirely (dictation, memo, self-filter).
5+
//! 2. Allowlist hit → named banner with the app's display name.
6+
//! 3. Otherwise → generic "Meeting detected" banner.
7+
//!
8+
//! The "bundle" string the matcher looks up on macOS is the bundle
9+
//! identifier returned by CoreAudio (`us.zoom.xos`). String comparison
10+
//! is case-insensitive, matching macOS filesystem / bundle-ID semantics.
11+
12+
/// A canonical "which meeting app is this?" identity. Platform-specific
13+
/// aliases map into one of these. The enum is cross-platform by design
14+
/// so follow-up PRs enabling Windows / Linux can extend the alias
15+
/// tables without touching the matching logic.
16+
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
17+
pub enum App {
18+
Zoom,
19+
Teams,
20+
Webex,
21+
FaceTime,
22+
Discord,
23+
Slack,
24+
/// Any browser — we can't tell which tab is open without reading URLs,
25+
/// and the whole point of Phase 1 was to avoid that. Shows as a
26+
/// generic "a browser meeting" banner.
27+
Browser,
28+
}
29+
30+
impl App {
31+
pub const fn display_name(self) -> &'static str {
32+
match self {
33+
App::Zoom => "Zoom",
34+
App::Teams => "Microsoft Teams",
35+
App::Webex => "Webex",
36+
App::FaceTime => "FaceTime",
37+
App::Discord => "Discord",
38+
App::Slack => "Slack",
39+
App::Browser => "a browser meeting",
40+
}
41+
}
42+
43+
/// Lower rank = higher priority. Used when multiple apps hold the
44+
/// mic simultaneously to pick the most meeting-ish candidate.
45+
const fn priority(self) -> u8 {
46+
match self {
47+
App::Zoom => 0,
48+
App::Teams => 1,
49+
App::Webex => 2,
50+
App::FaceTime => 3,
51+
App::Discord => 4,
52+
App::Slack => 5,
53+
App::Browser => 6,
54+
}
55+
}
56+
}
57+
58+
const ALIASES: &[(&str, App)] = &[
59+
("us.zoom.xos", App::Zoom),
60+
("com.microsoft.teams2", App::Teams),
61+
("com.microsoft.teams", App::Teams),
62+
("com.cisco.webexmeetingsapp", App::Webex),
63+
("com.webex.meetingmanager", App::Webex),
64+
("com.apple.FaceTime", App::FaceTime),
65+
("com.hnc.Discord", App::Discord),
66+
("com.tinyspeck.slackmacgap", App::Slack),
67+
("com.google.Chrome", App::Browser),
68+
("com.google.Chrome.canary", App::Browser),
69+
("com.apple.Safari", App::Browser),
70+
("org.mozilla.firefox", App::Browser),
71+
("company.thebrowser.Browser", App::Browser),
72+
("com.microsoft.edgemac", App::Browser),
73+
("com.brave.Browser", App::Browser),
74+
("com.operasoftware.Opera", App::Browser),
75+
("com.vivaldi.Vivaldi", App::Browser),
76+
];
77+
78+
const DEFINITELY_NOT_MEETINGS: &[&str] = &[
79+
// Self. Must stay in sync with `identifier` in tauri.conf.json.
80+
// If you fork and change the bundle ID, add your variant here.
81+
"com.meetily.ai",
82+
"com.meetily.ai.dev",
83+
"com.meetily.ai.debug",
84+
// System dictation / voice memos
85+
"com.apple.VoiceMemos",
86+
"com.apple.dictation",
87+
"com.apple.SpeechRecognitionCore",
88+
"com.apple.siri",
89+
"com.apple.assistantd",
90+
// Third-party dictation / transcription
91+
"will.flow.Wispr",
92+
"com.aliveseven.superwhisper",
93+
"com.chenyu.macwhisper",
94+
"com.flow.wispr",
95+
// Screen recorders
96+
"com.obsproject.obs-studio",
97+
"com.loom.desktop",
98+
"com.screenflow.ScreenFlow10",
99+
"com.screenflow.ScreenFlow11",
100+
];
101+
102+
fn keys_match(a: &str, b: &str) -> bool {
103+
a.eq_ignore_ascii_case(b)
104+
}
105+
106+
fn lookup(bundle_id: &str) -> Option<App> {
107+
ALIASES
108+
.iter()
109+
.find(|(alias, _)| keys_match(alias, bundle_id))
110+
.map(|(_, app)| *app)
111+
}
112+
113+
/// True if the bundle should be suppressed entirely (no banner, ever).
114+
pub fn is_blocked(bundle_id: &str) -> bool {
115+
DEFINITELY_NOT_MEETINGS
116+
.iter()
117+
.any(|blocked| keys_match(blocked, bundle_id))
118+
}
119+
120+
/// True if the bundle is in the curated allowlist of known meeting apps.
121+
/// Detection uses a shorter sustain threshold for these — we're confident
122+
/// it's a meeting app, so the longer flicker guard is unnecessary.
123+
pub fn is_known(bundle_id: &str) -> bool {
124+
lookup(bundle_id).is_some()
125+
}
126+
127+
/// Priority rank. Known apps return their `App::priority()` (0..=6),
128+
/// unknown apps return `u16::MAX`. Lower rank = higher priority.
129+
pub fn priority_of(bundle_id: &str) -> u16 {
130+
lookup(bundle_id)
131+
.map(|app| app.priority() as u16)
132+
.unwrap_or(u16::MAX)
133+
}
134+
135+
/// Human-friendly name for the banner. Unknown apps get the generic label.
136+
pub fn display_name(bundle_id: &str) -> &'static str {
137+
lookup(bundle_id)
138+
.map(App::display_name)
139+
.unwrap_or("a meeting")
140+
}
141+
142+
/// Pick the highest-priority non-blocked bundle from a list of active
143+
/// mic-holders. Known apps beat unknown apps; ties broken by input order.
144+
pub fn pick_best<'a, I>(active: I) -> Option<&'a str>
145+
where
146+
I: IntoIterator<Item = &'a str>,
147+
{
148+
let mut best: Option<(&str, u16)> = None;
149+
for bundle in active {
150+
if is_blocked(bundle) {
151+
continue;
152+
}
153+
let rank = lookup(bundle)
154+
.map(|app| app.priority() as u16)
155+
.unwrap_or(u16::MAX);
156+
match best {
157+
None => best = Some((bundle, rank)),
158+
Some((_, r)) if rank < r => best = Some((bundle, rank)),
159+
_ => {}
160+
}
161+
}
162+
best.map(|(b, _)| b)
163+
}
164+
165+
#[cfg(test)]
166+
mod tests {
167+
use super::*;
168+
169+
#[test]
170+
fn display_name_unknown_app_generic() {
171+
assert_eq!(display_name("com.unknown.niche-meeting-app"), "a meeting");
172+
}
173+
174+
#[test]
175+
fn pick_best_empty() {
176+
let active: Vec<&str> = vec![];
177+
assert_eq!(pick_best(active.iter().copied()), None);
178+
}
179+
180+
#[test]
181+
fn app_display_names_are_stable() {
182+
assert_eq!(App::Zoom.display_name(), "Zoom");
183+
assert_eq!(App::Teams.display_name(), "Microsoft Teams");
184+
assert_eq!(App::Browser.display_name(), "a browser meeting");
185+
}
186+
187+
#[test]
188+
fn app_priority_ordering() {
189+
assert!(App::Zoom.priority() < App::Teams.priority());
190+
assert!(App::Teams.priority() < App::Browser.priority());
191+
}
192+
193+
#[test]
194+
fn blocks_meetily_itself() {
195+
assert!(is_blocked("com.meetily.ai"));
196+
}
197+
198+
#[test]
199+
fn blocks_dictation_apps() {
200+
assert!(is_blocked("com.apple.VoiceMemos"));
201+
assert!(is_blocked("will.flow.Wispr"));
202+
assert!(is_blocked("com.aliveseven.superwhisper"));
203+
}
204+
205+
#[test]
206+
fn blocks_are_case_insensitive() {
207+
assert!(is_blocked("COM.MEETILY.AI"));
208+
assert!(is_blocked("com.Apple.VoiceMemos"));
209+
}
210+
211+
#[test]
212+
fn does_not_block_meeting_apps() {
213+
assert!(!is_blocked("us.zoom.xos"));
214+
assert!(!is_blocked("com.google.Chrome"));
215+
assert!(!is_blocked("com.unknown.app"));
216+
}
217+
218+
#[test]
219+
fn display_name_known_app() {
220+
assert_eq!(display_name("us.zoom.xos"), "Zoom");
221+
assert_eq!(display_name("com.microsoft.teams2"), "Microsoft Teams");
222+
}
223+
224+
#[test]
225+
fn display_name_browsers_generic() {
226+
assert_eq!(display_name("com.google.Chrome"), "a browser meeting");
227+
assert_eq!(display_name("com.apple.Safari"), "a browser meeting");
228+
}
229+
230+
#[test]
231+
fn pick_best_prefers_zoom_over_chrome() {
232+
let active = ["com.google.Chrome", "us.zoom.xos"];
233+
assert_eq!(pick_best(active.iter().copied()), Some("us.zoom.xos"));
234+
}
235+
236+
#[test]
237+
fn pick_best_skips_blocked() {
238+
let active = ["com.meetily.ai", "us.zoom.xos"];
239+
assert_eq!(pick_best(active.iter().copied()), Some("us.zoom.xos"));
240+
}
241+
242+
#[test]
243+
fn pick_best_unknown_only() {
244+
let active = ["com.unknown.app1", "com.unknown.app2"];
245+
assert_eq!(pick_best(active.iter().copied()), Some("com.unknown.app1"));
246+
}
247+
248+
#[test]
249+
fn pick_best_all_blocked_returns_none() {
250+
let active = ["com.meetily.ai", "com.apple.VoiceMemos"];
251+
assert_eq!(pick_best(active.iter().copied()), None);
252+
}
253+
254+
#[test]
255+
fn is_known_recognises_known_apps() {
256+
assert!(is_known("us.zoom.xos"));
257+
assert!(is_known("com.google.Chrome"));
258+
assert!(!is_known("com.unknown.app"));
259+
}
260+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
//! Meeting auto-detection via mic-activity.
2+
//!
3+
//! Observes which non-Meetily apps hold the microphone and fires
4+
//! `MeetingDetected` / `MeetingEnded` notifications at the standard
5+
//! sustain/end-silence thresholds. See
6+
//! `docs/plans/2026-04-20-feat-detect-meeting-start-and-end-plan.md`.
7+
8+
pub mod commands;
9+
pub mod matcher;
10+
pub mod service;
11+
pub mod signals;
12+
pub mod state;
13+
pub mod types;
14+
15+
pub use service::{spawn, DetectionService};

0 commit comments

Comments
 (0)