Skip to content

Commit 22c32f6

Browse files
committed
Checkpoint: in-flight signal listener, sonicfield bridge, floating panel & mic tap
Captures ~68 files of work-in-progress (new signal_listener/sonicfield/raw_audio modules, macOS FloatingPanel + MicTapManager + FloatingListenerView, project meta files) as a clean base before the hmm→oída rename.
1 parent 6a17b42 commit 22c32f6

68 files changed

Lines changed: 6308 additions & 5279 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
1-
AEAR_ENGINE_PROFILE=stub
1+
HMM_ENGINE_PROFILE=mac-mps
2+
HMM_AUDIO_DIR=~/Documents/hmm/audio
3+
HMM_MOSS_PREWARM=1
4+
HMM_SONICFIELD_ROOT=~/Documents/sonicfield
25
AEAR_SERVER_URL=http://127.0.0.1:8765
3-
AEAR_AKOUO_ROOT=/path/to/akouo
6+
AEAR_AKOUO_ROOT=~/Documents/sfl/akouo
47
AEAR_MOSS_AUDIO_REPO=./MOSS-Audio
58
AEAR_MOSS_INSTRUCT_MODEL=./weights/MOSS-Audio-4B-Instruct
69
AEAR_MOSS_THINKING_MODEL=./weights/MOSS-Audio-4B-Thinking

CHANGELOG.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Changelog
2+
3+
## 0.2.0 - Unreleased
4+
5+
- Made `mac-mps` the default engine everywhere (daemon, macOS shell, env
6+
examples); the daemon prewarms MOSS-Audio-4B-Instruct in the background and
7+
reports readiness at `/engine/status` (`HMM_MOSS_PREWARM=0` disables).
8+
- Added the deterministic **signal listener** (`aear/signal_listener.py`):
9+
DSP-only classification (silence / speech-like / music-like / tonal /
10+
percussive / noise / ambient), honest captions, and inferred hypotheses with
11+
numeric bases. Perception never falls back to placeholder text; without a
12+
model the evidence level honestly stays `measured_signal`.
13+
- Route presets now scope MOSS passes (`moss_passes`): Basic runs one caption
14+
pass (seconds, not minutes), Signal is DSP-only, Speech runs
15+
transcribe+speech, and the new **Deep** preset runs the full report.
16+
- Chunked mac-mps inference at 45 s per pass (`HMM_MOSS_CHUNK_SECONDS`), with a
17+
decode-degeneracy guard that discards token-soup output instead of letting it
18+
become claims, and a tokenizer-safe decode fallback.
19+
- Added Sonic Field bridge (`aear/sonicfield.py`, `/sonicfield/*`): after a
20+
listen, "Explore in the wiki" searches the wiki/lexicon, topics, journal,
21+
93k-item library, paths, research, notes, and labs for related concepts,
22+
with taxonomy alias normalization and Finder reveal.
23+
- Added `/events/stream` (SSE) so every surface mirrors one daemon state, and
24+
`/background/capture-request` so the web dashboard can ask the native shell
25+
to capture system audio.
26+
- Default audio directory moved to `~/Documents/hmm/audio`
27+
(`HMM_AUDIO_DIR`); captures, uploads, and fixtures land there.
28+
- Rebuilt the dashboard as one Listen surface (source + presets + skill
29+
manager), claims rendered by AKOUO category, memory and history cards, and a
30+
soft-minimal visual system; removed the floating spectral agent, its
31+
settings, RMS/peak meters, and the task tabs.
32+
- Rebuilt the macOS shell: the main window embeds the dashboard (WKWebView) so
33+
web and app never diverge; the floating listener is a minimal always-on-top
34+
panel with live signal, latest reading, and one Listen action; global
35+
hotkeys default to ⌃⌥L (listen) and ⌃⌥H (show/hide listener); the shell
36+
auto-starts the daemon and system tap and claims dashboard capture requests.
37+
38+
## 0.1.0 - Unreleased
39+
40+
- Added platform data directory support through `HMM_DATA_DIR` / `AEAR_DATA_DIR`
41+
with macOS default `~/Library/Application Support/hmm`.
42+
- Disabled implicit Hugging Face hub model fallback unless explicitly enabled.
43+
- Added bearer-token enforcement for wildcard/LAN daemon binds.
44+
- Added raw-audio status and wipe endpoints for local upload/live-buffer
45+
retention, including pre-data-dir checkout `uploads/` recordings via the
46+
`include_legacy` wipe option (the dashboard Wipe raw audio button sets it).
47+
- Bounded in-memory stopped live sessions to the most recent few.
48+
- Added Earworm-compatible Akousmata trace/session/context metadata.
49+
- Updated AKOUO harness routing for the v0.5 public command contract and
50+
derived evidence levels.
51+
- Reduced long-audio memory pressure by bounded DSP inspection and per-chunk
52+
source reads.
53+
- Added release/security/contribution/citation metadata.

CITATION.cff

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
cff-version: 1.2.0
2+
title: "hmm: local listening agent for the Listening Stack"
3+
message: "If you use hmm in research, please cite the software and the Listening Stack paper."
4+
type: software
5+
authors:
6+
- name: "Sonic Field Labs"
7+
version: 0.1.0
8+
date-released: 2026-07-02
9+
license: Apache-2.0
10+
repository-code: "https://github.com/sonicfieldlabs/hmm"
11+
preferred-citation:
12+
type: article
13+
title: "The Listening Stack: Agentic Sonic Computation for Hybrid Listening Ecologies"
14+
authors:
15+
- family-names: "Isaza"
16+
given-names: "eme"
17+
year: 2026

CONTRIBUTING.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Contributing
2+
3+
`hmm` is a local-first listening agent. Changes should preserve four contracts:
4+
5+
- Raw audio never leaves the operator machine by default.
6+
- Raw audio retention and deletion paths stay explicit and testable.
7+
- MOSS-Audio output is perception evidence, not final truth.
8+
- AKOUO claim categories remain separate: heard, measured, inferred,
9+
interpreted, speculative, undetermined.
10+
11+
## Local Checks
12+
13+
```bash
14+
uv sync --extra dev
15+
uv run python -m unittest discover -s tests
16+
uv run pytest
17+
```
18+
19+
For release smoke checks, start the stub daemon and run:
20+
21+
```bash
22+
scripts/run_local_checks.sh release
23+
```
24+
25+
## Dependencies
26+
27+
Keep runtime dependencies small. Network-capable packages should be dev-only
28+
unless the daemon itself needs them at runtime.

LICENSE

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,3 +155,19 @@ such Contributor by reason of your accepting any such warranty or additional
155155
liability.
156156

157157
END OF TERMS AND CONDITIONS
158+
159+
APPENDIX: How to apply the Apache License to your work.
160+
161+
Copyright 2026 Sonic Field Labs
162+
163+
Licensed under the Apache License, Version 2.0 (the "License");
164+
you may not use this file except in compliance with the License.
165+
You may obtain a copy of the License at
166+
167+
http://www.apache.org/licenses/LICENSE-2.0
168+
169+
Unless required by applicable law or agreed to in writing, software
170+
distributed under the License is distributed on an "AS IS" BASIS,
171+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
172+
See the License for the specific language governing permissions and
173+
limitations under the License.

NOTICE

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
hmm
2+
Copyright 2026 Sonic Field Labs
3+
4+
This product includes software developed by Sonic Field Labs.
5+
6+
hmm integrates with MOSS-Audio model code and weights when the operator installs
7+
them separately. MOSS-Audio is developed by OpenMOSS / MOSI.AI and is not
8+
redistributed in this repository.
9+
10+
AKOUO and Earworm are Sonic Field Labs listening-stack companion projects. Their
11+
local checkouts or released packages are referenced as optional integration
12+
surfaces; this repository embeds only hmm-specific glue code and schemas.

PLAN.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
1-
# AEAR Implementation Plan
1+
# hmm Implementation Plan (historical AEAR phases)
22

3-
This file tracks the `mossagent.md` plan as implemented in this repository.
3+
This file tracks the `mossagent.md` plan as implemented in this repository,
4+
written under the project's earlier name AEAR and kept as a phase log. For the
5+
current architecture map, see `docs/architecture/current-state.md`.
46

57
## Phase 0 - Feasibility Spike
68

README.md

Lines changed: 44 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -50,16 +50,19 @@ New user-facing commands and UI copy use lowercase `hmm`.
5050

5151
## Quick Start
5252

53+
Prerequisites: Python 3.12+, `uv`, and `ffmpeg` for non-WAV uploads or browser
54+
recordings.
55+
5356
Install dependencies:
5457

5558
```bash
56-
uv sync
59+
uv sync --extra dev
5760
```
5861

59-
Run the daemon in development mode with the local stub engine:
62+
Run the daemon (MOSS-Audio on Apple Silicon is the default profile):
6063

6164
```bash
62-
uv run hmm --profile stub --host 127.0.0.1 --port 8765
65+
uv run hmm --host 127.0.0.1 --port 8765 # add --profile stub for a model-free dev run
6366
```
6467

6568
Open the dashboard:
@@ -71,9 +74,10 @@ http://127.0.0.1:8765
7174
Generate a normalized listening event:
7275

7376
```bash
77+
uv run python -c "import numpy as np, soundfile as sf; t=np.arange(16000)/16000; sf.write('/tmp/hmm-tone.wav',(0.15*np.sin(2*np.pi*440*t)).astype('float32'),16000)"
7478
curl -s http://127.0.0.1:8765/listen-event \
7579
-H 'content-type: application/json' \
76-
-d '{"path":"MOSS-Audio/test/test_en.mp3","route_preset":"basic"}'
80+
-d '{"path":"/tmp/hmm-tone.wav","route_preset":"basic"}'
7781
```
7882

7983
Run a routed local session and write `sessions/<stamp>-<slug>/`:
@@ -112,8 +116,8 @@ apps/macos/script/build_and_run.sh
112116

113117
The shell stages `apps/macos/dist/hmm.app`, adds a menu bar extra, opens the
114118
dashboard, triggers background quick capture, and exposes an optional global
115-
hotkey in Settings. If the daemon is offline, the shell can start a local
116-
development daemon with `uv run hmm --profile stub`. Launch at login is
119+
hotkey in Settings. If the daemon is offline, the shell starts one automatically with the
120+
`mac-mps` profile. Launch at login is
117121
available in Settings and is never enabled automatically.
118122

119123
The shell also includes an explicit native system-output signal tap. It uses
@@ -127,6 +131,13 @@ Native source-route profiles are exposed at `/native/system-audio/routes`; the
127131
current route is `display_mix`, meaning the selected display's system mix with
128132
the hmm process excluded.
129133

134+
Raw browser uploads and live chunks are stored under the configured hmm data
135+
directory, not the source checkout. Inspect them with `/raw-audio/status` and
136+
delete them with `/raw-audio/wipe`; the dashboard exposes this as Wipe raw
137+
audio. Recordings written by older builds into the checkout's `uploads/` are
138+
reported under the status `legacy_*` fields and deleted when the wipe request
139+
sets `include_legacy` (the dashboard button does).
140+
130141
Package a local unsigned app archive:
131142

132143
```bash
@@ -153,10 +164,10 @@ Existing listening events can be rerun through a different preset with
153164
new audio. Rerun responses include a conservative route comparison over route
154165
ids, summary changes, warnings, deterministic DSP deltas, and applied comparison
155166
filters. The daemon also keeps a bounded recent-result list at
156-
`/background/history`, persists derived event JSON to
157-
`sessions/recent-results.json`, and excludes incognito events from that durable
158-
history by default. Pinned recent results are persisted separately from the
159-
rolling recent list, and `/background/history/export`, `/background/history/pin`,
167+
`/background/history`, persists derived event JSON under the configured hmm data
168+
directory, and excludes incognito events from that durable history by default.
169+
Pinned recent results are persisted separately from the rolling recent list, and
170+
`/background/history/export`, `/background/history/pin`,
160171
`/background/history/batch-pin`, `/background/history/archive`, and
161172
`/background/history/clear` expose derived-history export, pin/unpin, batch
162173
pinning, archive-to-file, and explicit clear controls without copying raw audio.
@@ -236,7 +247,8 @@ left for the desktop app-shell phase.
236247
Run tests that do not require downloaded model weights:
237248

238249
```bash
239-
python3 -m unittest discover -s tests
250+
uv run python -m unittest discover -s tests
251+
uv run pytest
240252
```
241253

242254
## Real MOSS-Audio Profile
@@ -269,6 +281,12 @@ and sets `audio_input_mask = input_ids == processor.audio_token_id`.
269281
`AEAR_MOSS_RESIDENT=single` hot-swaps Instruct and Thinking instead of keeping
270282
both 4B models resident.
271283

284+
`hmm` will not silently download model code or weights. If the local `weights/`
285+
paths are absent, the `mac-mps` profile falls back to the stub engine unless
286+
`AEAR_REQUIRE_MODEL=1` is set, and Hugging Face hub lookup is refused unless
287+
`HMM_ALLOW_HF_HUB=1` or `AEAR_ALLOW_HF_HUB=1` is set. `HF_HUB_OFFLINE=1` always
288+
keeps hub lookup disabled.
289+
272290
## CUDA/SGLang Profile
273291

274292
Start the official MOSS-Audio SGLang fork separately, then point `hmm` at it:
@@ -283,12 +301,21 @@ Thinking budgets are forwarded through `custom_params.thinking_budget`.
283301
## Privacy Defaults
284302

285303
`hmm` is local-first. The daemon binds to `127.0.0.1` by default and does not
286-
upload audio. Background-style buffers are ephemeral by product policy, but the
287-
current browser live path still writes temporary chunks to local `uploads/`
288-
because browser `MediaRecorder` sends encoded files to the daemon. Quick live
289-
captures are labeled with `raw_audio_policy: temp`. Native system-output temp
290-
captures use a separate retention policy under `native_temp_audio_retention`
291-
and cleanup only targets files matching the native capture naming pattern.
304+
upload audio. If you bind to `0.0.0.0` or `::`, the daemon refuses to start
305+
unless `HMM_AUTH_TOKEN` or `AEAR_AUTH_TOKEN` is set; clients then send
306+
`Authorization: Bearer <token>`.
307+
308+
Persistent local data defaults to `~/Library/Application Support/hmm` on macOS,
309+
or `$XDG_DATA_HOME/hmm` / `~/.local/share/hmm` elsewhere. Override it with
310+
`HMM_DATA_DIR` or `AEAR_DATA_DIR`.
311+
312+
Background-style buffers are ephemeral by product policy, but browser live
313+
capture still writes temporary chunks to the data-dir `uploads/` because
314+
`MediaRecorder` sends encoded files to the daemon. Quick live captures are
315+
labeled with `raw_audio_policy: temp`. Native system-output temp captures use a
316+
separate retention policy under `native_temp_audio_retention`, raw uploads use
317+
`upload_audio_retention`, and `/raw-audio/wipe` removes all upload/live-buffer
318+
raw audio on request.
292319

293320
Akousmata memory is explicit: events are saved only when the user calls
294321
`/memory/remember` or uses the dashboard remember action. File-based listening

SECURITY.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Security Policy
2+
3+
## Supported Versions
4+
5+
Security fixes are accepted against the current `main` branch until the first
6+
public release tag is cut.
7+
8+
## Reporting A Vulnerability
9+
10+
Please report security issues privately to Sonic Field Labs before public
11+
disclosure. Include the affected commit, local configuration, reproduction steps,
12+
and whether raw audio or private listening traces can be exposed.
13+
14+
## Local-First Boundaries
15+
16+
`hmm` binds to `127.0.0.1` by default. Wildcard or LAN binds are refused unless
17+
`HMM_AUTH_TOKEN` or `AEAR_AUTH_TOKEN` is set, and token-protected clients must
18+
send `Authorization: Bearer <token>`.
19+
20+
MOSS-Audio Hugging Face model lookup is disabled by default. Download weights
21+
into `weights/` or set `HMM_ALLOW_HF_HUB=1` / `AEAR_ALLOW_HF_HUB=1` explicitly.
22+
`HF_HUB_OFFLINE=1` always disables hub lookup.

0 commit comments

Comments
 (0)