feat(detection): auto-detect meetings via microphone activity (macOS) - #34
Closed
AzimovS wants to merge 12 commits into
Closed
feat(detection): auto-detect meetings via microphone activity (macOS)#34AzimovS wants to merge 12 commits into
AzimovS wants to merge 12 commits into
Conversation
Observes which non-Meetily apps hold the default input device and fires MeetingDetected / MeetingEnded banners at configurable thresholds — no new permissions, no browser URL reading, no window-title parsing. Matches the approach used by char (fastrepl/char) and Granola. - detection/ module with poll-based service (1s cadence) driving a state machine: Idle → Sustaining → Detected → Ending → Idle - macOS sampler via cidre: DeviceIsRunningSomewhere gates per-process enumeration (kAudioProcessPropertyIsRunningInput, bundle_id) - Hybrid allowlist/blocklist matcher: named apps (Zoom/Teams/browsers) get their display name; unknown apps fire a generic banner; Meetily, dictation tools, screen recorders suppressed - 10s sustain for known apps, 30s for unknown; 30s end-silence; no automatic dismissal on natural end (only explicit user dismiss) - Notification gated on audio::recording_commands::IS_RECORDING, not the legacy RECORDING_FLAG (which the UI recording path never sets) - Windows/Linux stub returns an empty snapshot — Phase 2 work - 22 unit tests for state transitions, blocklist matching, display-name resolution Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prepares for Phase 2 Windows/Linux samplers. The flat KNOWN_MEETING_APPS/DEFINITELY_NOT_MEETINGS tuples would have needed duplication for each platform's identifier flavor (bundle ID vs exe basename vs binary name); replacing with an App enum + cfg-gated ALIASES tables keeps the matching logic platform-agnostic. - App enum (Zoom/Teams/Webex/FaceTime/Discord/Slack/Browser) holds canonical display name + priority rank - Per-platform ALIASES: macOS bundle IDs, Windows exe names, Linux process binaries - Per-platform DEFINITELY_NOT_MEETINGS with platform-appropriate self/dictation/recorder identifiers - keys_match() encapsulates case-insensitive comparison on macOS/Windows (filesystem semantics) vs case-sensitive on Linux (process names) - Tests split into per-platform cfg modules; all Phase 1 macOS tests retained; added Windows + Linux alias/block tests Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the no-op stub on Windows with a real IAudioSessionManager2 session enumerator. Matches the cadence and contract of the macOS sampler: polled by the detection service, returns a MicSnapshot of exe basenames for processes currently transmitting to the default capture endpoint. Algorithm (per tick): - CoInitializeEx (thread-local, MTA, once per worker thread) - CoCreateInstance IMMDeviceEnumerator - GetDefaultAudioEndpoint(eCapture) - IAudioSessionManager2 → GetSessionEnumerator → iterate - Filter to AudioSessionStateActive - Skip own PID and PID 0 (system session) - OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION) + QueryFullProcessImageNameW → basename → active_bundles Graceful degradation: if WASAPI init fails at construction (headless CI, broken audio subsystem), factory falls back to stub sampler so the service keeps running. Same behaviour retrofitted for macOS. Adds windows 0.58 dep under [target.'cfg(target_os = "windows")']. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the no-op stub on Linux with a real PulseAudio introspection sampler. Works with both native PulseAudio and PipeWire's pulse-compat layer (default on all major distros: Ubuntu 23.04+, Fedora 34+, etc). Architecture: - Dedicated native thread owns the Mainloop + Context (both are !Send and must stay on one thread). - Shared snapshot via Arc<Mutex<Vec<String>>>; tokio-side snapshot() just clones the current contents. - Poll get_source_output_info_list at 1s cadence on the pulse thread. - Self-filter by std::process::id() via application.process.id proplist. - Bundle key = application.process.binary (falls back to .name). - Reconnect with backoff 1s→10s on State::Failed / State::Terminated; factory falls back to stub if initial connect times out at 3s. Adds libpulse-binding 2.28 dep under [target.'cfg(target_os = "linux")']. Limitation: users on pure-PipeWire systems without pulse-compat (rare) get no detection. App starts normally; detection silently inactive. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds lockfile entries for windows 0.58 (Win32 feature set) and libpulse-binding 2.28. Generated by cargo check --lib. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…face Resolves review findings 003, 004, 006, 011, 012, 013, 014, 015, 016, 017. Lifecycle (003): - DetectionService::shutdown() signals poll loop to exit at next tick - Called from RunEvent::Exit before tauri starts dropping AppHandle / state. Removes the std::mem::forget(running) smell. Dependency inversion (015): - Audio layer calls DetectionService::set_recording(true/false) at each IS_RECORDING.store site instead of detection reaching back into audio::recording_commands. Advance() no longer takes is_recording param. State machine UX (011, 012): - Sustaining now upgrades to a strictly-higher-priority candidate when one arrives mid-sustain (Chrome → Zoom handoff case). - Ending requires `ending_reacquire_confirm` (3s) of continuous mic activity before cancelling the silence counter. Absorbs sub-3s flicker without spurious MeetingEnded. Agent-native + observability (013, 016): - Emit `meeting-detected` / `meeting-ended` Tauri events alongside banner delivery. DetectedMeeting is already Serialize, so the payload is the canonical struct. - New Tauri commands: `dismiss_detected_meeting(bundle_id)` and `get_detection_state()` returning DetectorPhaseSnapshot with phase name, display name, elapsed/remaining ms, and recording state. Memory (006): - DetectorState reaps expired `dismissed` entries every 60 advances (~1 min). HashMap no longer grows unbounded. Privacy (004): - Bundle IDs / raw process identifiers demoted from info! to debug! everywhere in the detection hot path. Display names (already sanitized — unknown apps say "a meeting") stay at info. Dismiss subsystem (017): - Not deleted — wired end-to-end via the new Tauri commands and the TTL reaper. Phase 4 "ignore this app" UI now has the Rust-side API it needs with no further backend work. Trait naming (014 partial): - Left SignalSampler name alone for now. Documented the pull-vs-event-driven concern in the plan; future change if another signal source appears. Tests: 40 detection tests pass (added sustaining-priority-upgrade, sub-3s flicker guard, phase_snapshot, reaper). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves review findings 001, 002, 007. Operation lifecycle (001): Previously `collect_snapshot` acquired `mainloop.lock()`, issued the introspect op, released the lock, and let the Operation drop when it fell out of scope — outside the lock. libpulse-binding's documentation requires the mainloop lock to be held during Operation drop (pa_operation_unref), or else the pulse thread can race with the operation's internal state. Now: - Op is created under lock - Lock is released while we wait for the callback - Lock is re-acquired to drop the Op (→ pa_operation_unref) Busy-wait removal (007): Replaced the 10ms sleep-poll for `done.load()` with a std::sync::mpsc sync_channel(1). Callback's End/Error arms `try_send(())`; the main thread blocks on `recv_timeout(2s)`. Cuts median wait from ~15ms (CLOCK_REALTIME granularity) down to the actual callback completion time (~1-5ms). Init timeout cleanup (002): Previously the timeout branch set shared.shutdown=true and returned without joining, leaking the worker thread. Now: poll-wait up to 2s for `handle.is_finished()`, join if it exits, otherwise log a warn and detach (thread terminates on next shutdown flag check anyway). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…gnostic Resolves review findings 008, 009, 010. Sidecar self-filter (009): Added llama-helper + ffmpeg to Windows and Linux DEFINITELY_NOT_MEETINGS so that Meetily-spawned sidecars — current or re-enabled later — don't leak into the detector's snapshot and fire self-banners. These are the executables in tauri.conf.json's externalBin set. Dev bundle variants (010): Added com.meetily.ai.dev and com.meetily.ai.debug to the macOS blocklist so signed dev builds and debug builds self-filter correctly. Doc note on matcher.rs points forks at the entries they need to update if they rename the bundle. EDR diagnostic (008): Windows OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION) can be denied for EDR-protected processes (Defender for Endpoint, CrowdStrike, SentinelOne). Previously the denial was silently dropped, and users on those stacks saw the feature as permanently broken with no log trail. Now: warn once per unique PID when OpenProcess returns ACCESS_DENIED, with a hint that EDR is the likely cause. No log spam (warn only fires on first-seen PID) and no PII (only the non-identifying integer PID is logged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves review findings 019, 020 (partial). Linux reconnect backoff (019): Added ±20% jitter to the pulseaudio reconnect backoff to avoid thundering-herd if multiple Meetily instances or users on a shared host reconnect in lockstep after a daemon restart. Uses the workspace's already-present rand 0.8.5. macOS gate error log (020): The DeviceIsRunningSomewhere property read previously swallowed errors via unwrap_or(false). If the subsystem gets wedged we'd degrade to "always idle" with zero diagnostics. Now logs at trace so a developer with RUST_LOG=trace can see the breadcrumb. SessionOutcome collapse (018): considered and deferred — the semantic clarity of Connected vs ConnectFailed outweighs the ~15 LOC savings. Remaining 020 items (over-documented comments, per-tick Arc churn, Phase::clone, Windows 512-wchar buffer) are cosmetic and deferred. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
Author
Review findings addressedMulti-agent code review surfaced 20 findings; 16 resolved in this branch. P1 fixed — all four
P2 fixed — 10 of 12
P3 fixed
Deferred (low value for v1)
Test status
Branch commits |
libpulse-binding's threaded Mainloop::lock / unlock take &mut self. collect_snapshot took &Mainloop so the four lock/unlock calls failed with E0596 on linux targets. Bump the signature to &mut Mainloop and pass the owned-mut from run_session. Caught by CI; untestable from macOS because the module is #[cfg(target_os = "linux")] gated.
QueryFullProcessImageNameW and PROCESS_NAME_WIN32 live in windows::Win32::System::Threading in windows 0.58, not ProcessStatus (which only exposes K32*-prefixed PSAPI helpers in this version). Move both imports and drop the now-unused Win32_System_ProcessStatus feature. Verified against the local registry source.
macOS is the only platform the author has validated end-to-end. The Windows (WASAPI) and Linux (libpulse-binding) samplers compile and remain in the tree — only the spawn call, Tauri command registrations, and shutdown hook are cfg-gated so the detection service never runs on those platforms. The sampler modules are already `#[cfg(target_os = ...)]`-gated; this change ensures `DetectionService` is never constructed on Win/Linux, so `notify_detection_recording_state` in recording_commands silently no-ops via `try_state` returning None. Follow-up PRs per platform will flip these cfgs once each is validated on real hardware and a settings-level kill switch is in place.
AzimovS
changed the title
Owner
Author
|
Superseded by #35 — a clean-cut macOS-only subset of this work. The Windows (WASAPI) and Linux (libpulse-binding) samplers developed on this branch are preserved in the Closing in favour of #35. Reasoning: shipping validated macOS code separately from unvalidated Win/Linux code avoids conflating confidence levels, and users of those platforms don't have an in-app way to disable detection today if something goes wrong. Phase 2 will land per-platform once a settings-level kill switch is in place. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds meeting auto-detection via microphone activity on macOS. When a non-Meetily app holds the default input device, a banner fires at a configurable sustain threshold. When the mic is released during an active recording, an "Meeting ended — tap to stop recording" banner fires. Matches the approach used by
char(fastrepl/char) and Granola — no browser URL reading, no window title parsing, no new permissions.Scope: macOS only
The Windows (WASAPI
IAudioSessionManager2) and Linux (libpulse-binding) samplers are in the tree, compile green on CI, and are ready to ship — but neither has been validated on real hardware by the author. Thedetection::spawncall, Tauri command registrations, and shutdown hook are#[cfg(target_os = "macos")]-gated so nothing runs on Windows/Linux. Those platforms will be enabled in follow-up PRs once each is validated end-to-end and a settings-level kill switch is in place (see "Follow-up work" below).Why: if a platform-specific bug crashed the app on launch, users today have no in-app way to disable detection (
show_meeting_detected/show_meeting_endedprefs only suppress the banner — the poll loop keeps running). Shipping all three platforms simultaneously conflates validated macOS code with compile-green-but-unexercised Win/Linux code. Gating keeps the work in-branch without taking on that risk.Commits
341c04fmacOS via CoreAudio (Phase 1, shipped and validated end-to-end)f448681matcher refactor:Appenum + per-platform alias tablesedd1424Windows via WASAPIIAudioSessionManager2(built, not enabled)de84d74Linux vialibpulse-bindingon a dedicated thread (built, not enabled)bc76a58fix: linux Mainloop requires&mutfor lock/unlock (CI-caught)ed812eafix: windows imports —QueryFullProcessImageNameWlives inThreading, notProcessStatus(CI-caught)81c3beccfg-gate Win/Linux spawn to defer enablement until validatedDetection thresholds (macOS)
After natural meeting-end the state machine returns cleanly to Idle — no automatic 10-min dismissal (dismissal exists as an internal API for future "ignore this app" button).
macOS implementation
Uses
cidrecrate. On each tick:kAudioDevicePropertyDeviceIsRunningSomewhereas a cheap gateca::System::processes()and filter byis_running_input()ca::Process::bundle_id()Testing
#[cfg(...)]test modules cover macOS bundle-ID matching; Windows and Linux matcher tests also pass under their respective cfgs.Manual test matrix (macOS)
Post-deploy monitoring
infolevel, macOS only):detection: Idle → Sustaining(— a candidate crossed observation thresholddetection: Sustaining(...) → Detected— banner fireddetection: Ending(...) → Idle— natural meeting-end (with/without banner based on recording state)mic-activity snapshot failed— CoreAudio sampler errorMeeting detection disabled — failed to init mic-activity sampler— fallback to stub samplerFollow-up work
Phase 2a: Windows enablement
detection_enabledkill switch (settings-file gate checked beforespawn)#[cfg(target_os = "macos")]gate for Windows inlib.rsPhase 2b: Linux enablement
application.process.binarykey semantics differ#[cfg(target_os = "macos")]gate for Linux inlib.rsKill switch design (blocker for Phase 2):
detection_enabled: booltoNotificationSettings(defaulttrueon mac,falseon Win/Linux until validated per-user)detection::spawnand short-circuit if disablednotifications.json, setdetection_enabled: false, relaunch"Plan documents
docs/plans/2026-04-20-feat-detect-meeting-start-and-end-plan.md— overall plan (Phase 1 + scoping decisions)docs/plans/2026-04-20-feat-meeting-detection-phase-2-windows-linux-plan.md— detailed Phase 2 plan (now the roadmap for follow-up PRs)docs/brainstorms/2026-04-20-meeting-auto-detect-brainstorm.md— origin discussion🤖 Generated with Claude Code