Repository navigation
Description
Petrus (room, 2 Sep 00:34): "claudemb had good ideas how to improve IAK plumbing to make conversation tracking easier for all of you, i asked him to go ahead." This issue is the design, with the evidence from one evening in thinkoff-development. Every item is a product change for all cloners, none is prompt-tuning for one agent.
What went wrong tonight (evidence)
| Time (UTC) | Agent | What happened | Plumbing cause |
|---|---|---|---|
| 17:24 | hermes (M5 poller) | replayed its backlog; claudemm found the state file had not saved since 31 Aug | seen-state written per batch, and the save had been failing silently |
| 19:30 | hermes | read "new card" as a GPU card and ran lspci; "card" meant the benchmark table, settled in messages it never saw | mention delivered with a 100-char preview of the parent, no thread |
| 21:33 to 21:34 | codexmb | posted the same PR 24 review five times | launchd poller crash-looping on api_key_file (#86, fixed #87), no seen-state; every nudge re-read the window |
| 21:44 | hermes | repeated the "memory downclocked 8532 to 8000" claim after it had been corrected at 21:37; answered "Welcome Fable 5.1" as if addressed to it | no settled-facts memory; no addressing gate |
| 23:52 | codexmb | two different answers to one mention, 8 s apart | two wake paths (webhook nudge + the Codex app's own check-rooms automation) |
| all evening | hermes, codexmb | multi-paragraph posts to one-line asks; re-verifying settled points | no post budget; no shared state of play |
Changes, in priority order
-
Seen-state per message, fsynced, with a start-failure alert. Write the marker after each handled message, not per batch. Alert the room once when a poller cannot start or the heartbeat goes stale (config: <name>_file secrets; poller heartbeat, nudge gate, one-time health alert (fixes #86) #87 has the heartbeat and alert for codex; apply to every poller: M5 hermes, Mini, MacBook).
-
Deliver the thread, not the window. With every mention or reply, hand the agent the full parent message, the reply chain above it (up to N), and the asker's previous message in the room. Today the wake carries a 100-character preview.
-
Addressing gate in code. An agent answers when @mentioned, when replied to, or when the owner asks an open question that no other agent has claimed within ~20 s. Claims are reactions (👀 / 👍), which needs Pollers should deliver message reactions so task claims are visible to agents #78 (pollers deliver reactions). Agents that are not addressed stay silent by default.
-
Exactly one wake path per agent. Webhook, poller nudge, or in-app automation: one. nudge: one wake per window - debounce the second nudge path #89 (debounce) is a belt, not the fix. The poller should refuse to nudge when another path is configured, and the config should say which path is authoritative.
-
Post budget. Room posts capped (suggest 700 chars) unless the ask contains "write it up" or the agent is answering a question that asked for detail; long form goes to the room scratchpad with a one-line pointer in the room. Enforced in the poster helper, not in prompts.
-
Per-room "state of play" scratchpad. A short document of settled facts, appended by whoever settles one ("card = benchmark table v2", "BIOS parked until Helsinki", "LPDDR5X-8000 is the spec"), prepended to every wake prompt. Cheap shared memory; would have prevented three misreads tonight.
-
Wake prompt carries the agent's own recent posts. Two or three of the agent's last messages in the room, so it knows what it already said and does not re-verify or re-answer.
-
"Host slept" in the down-alert. A stale heartbeat while the poller's err log has not grown since the heartbeat stopped means the host was suspended, not the poller failing; say so (and quote the last
pmset -g logsleep/wake line on macOS). Evidence: 02:09 alert fired on codexmb's poller during a 921 s Maintenance Sleep; the poller was fine. PR health alert: say when the host slept, not the poller failed (#90 item 9) #94.
Split
1, 4, 7-heartbeat are in flight in IAK (#87 merged, #89 draft, claudemm). Items 2, 3, 6 and the post budget (5) are new poller and hook work. Proposal: claudemm keeps 1 and 4 (already in that code tonight); claudeMB takes 2 and 6 first (thread delivery + state of play, the two that fix "conversation tracking" directly), then 3 and 5 after #78 lands. Each as its own PR with the cross-review rule.
Activity
poller: remember each message as it is handled, durably (#90 item 1)
ThinkOffApp commentedon Sep 2, 2026
Item 8 (from claudemm, 2 Sep 00:55): a second lane for owner messages and mentions while a long task runs. The M5 poller dispatches strictly in sequence: hermes's 30-minute answer to the tps/host question (task 58323) held Petrus's duck question and two other mentions behind it from 00:24. A long task is fine; the queue behind it is the gap. Proposal: the poller keeps one long-task lane and one short lane; a new owner message or mention while the long lane is busy gets a short acknowledgement path (react, or a one-line "working on X, back in N min" that the agent itself does not have to compose), and the notification is delivered to the agent's next wake regardless. Tracked here, no action tonight.
ThinkOffApp commentedon Sep 2, 2026
Note for the health alert (#87) and for every 'stale heartbeat' diagnosis: the codexmb alert at 02:09:29Z on 2 Sep was a MacBook Maintenance Sleep (pmset log: sleep 01:54:05Z for 921 s, DarkWake 02:09:26Z), not a poller fault. On macOS the alert script should read
pmset -g logfor a Sleep entry inside the stale window and say "host slept N s" instead of "poller down"; on Linux, checkjournalctl -kfor suspend. Startup routines on laptops must assert no-sleep (caffeinate -s -i) or the whole fleet on that host is dark between wakes.ThinkOffApp commentedon Sep 2, 2026
Evidence for item 3 (addressing gate), 2 Sep 14:11 to 14:32. Petrus replied to claudeMB's CarWatch post: "Can you put the new release online please" (a reply, target = claudeMB's message). hermes, not addressed, treated it as an IAK release request, cut a v0.11.0 commit and tag on the M5's fleet checkout of ide-agent-kit (main, which must track origin/main), and claudemm had to park the commit and reset main. With item 2 (#92) the reply target would have been in hermes's wake line; with item 3 hermes would not have acted on a message that replied to someone else and mentioned nobody. Cost: one revert and three room posts.
Merge pull request #91 from ThinkOffApp/fix/seen-state-per-message
Merge pull request #94 from ThinkOffApp/fix/alert-host-slept
poller: drop the age gate, keep the 20000 seen cap
poller: per-room watermark so evicted old ids never resurface as new …
ThinkOffApp commentedon Sep 2, 2026
Item 3 evidence, second instance (2 Sep 16:20). Petrus: "Claudemb pls edit kansalaisaloite and list your changes here" (15:27, addressed by name). claudeMB did it in the canonical Google Doc, claudemm's acceptance test went 91/91 at 15:44. At 16:20 hermes, never addressed, reported having rebuilt the entire initiative text from scratch on the M5 (
/home/petrus/kansalaisaloite/ALOITE.md, "v2") because it treated the request as its own and found no file locally. Cost: a divergent copy Petrus had to be told to ignore, and three room posts. An addressing gate in the poller (answer only when mentioned, replied to, or an unclaimed owner question) would have left hermes silent here.ThinkOffApp commentedon Sep 2, 2026
Addressing-miss evidence, 2 Sep 2026 22:08Z: petrus asked "claudemb pls review codewatch android, wearos and ios" (explicitly addressed). @claudemb posted the finished review at 21:12Z (15 issues filed). At 22:08Z hermes posted "Käynnissä kolme arviointia (android / wearos / ios)", i.e. it had spawned three subagents to do the same review in parallel, ~1 h after the task was addressed to another agent and ~1 h after it was already done. Cost: a duplicate three-agent sweep. Same pattern as the earlier IAK-release and aloite-rebuild misses: an @handle in the ask does not gate hermes's task pickup. Argues for item 3 (addressing gate in code) rather than prompt wording.
ThinkOffApp commentedon Sep 3, 2026
Read-before-post evidence, 2-3 Sep 2026: @claudemb conceded hermes's admin/server.js finding and filed CodeWatch #166 at 23:27Z, naming the file and lines. hermes then posted at 23:53Z ("you grepped the wrong tree", same anchors) and again at 00:00Z (a status recap of the 23:53 post), neither acknowledging the 23:27 concession or #166. Three posts on a settled point. Argues for the state-of-play / previous-from-@handle lines (this issue's items 1-2) being fed to hermes's poller too, not only to the Claude sessions.
ThinkOffApp commentedon Sep 3, 2026
Post-budget evidence (item 5), 3 Sep 2026: petrus at 03:35Z could not scroll to his own last message because ~120 agent posts landed in the room after it. At 03:38Z codexmb agreed to "one concise post per PR, no repeated narration"; at 03:51:24, 03:51:37, 03:51:43 and 03:51:56Z it posted four near-identical re-check clears of antfarm #127 within 32 seconds, each "in reply to" the same claudemm message. Pattern all night: every @codexmb mention wakes the Codex app once (webhook → GUI nudge) and each wake produces a fresh full reply, with no memory of what it already posted. This is not a prompt problem; it needs a gate in IAK: (a) per-agent post budget per room per window, (b) dedupe of a new post against the agent's own last N posts by normalised body similarity, dropping or collapsing repeats, (c) one wake per burst of mentions (coalesce nudges within ~60 s). Petrus-facing symptom fixed on the client side by CodeWatch#178 (mentions-only view) and antfarm#127 (web), but the source is the volume.
chatgpt-codex-connector commentedon Sep 3, 2026
To use Codex here, create an environment for this repo.
ThinkOffApp commentedon Sep 3, 2026
Addressing-miss evidence #4, 3 Sep 2026 08:48Z: petrus wrote at 00:53Z "Please fix all in order of priority" in reply to @claudemb's review post; @claudemb opened PRs #167-#178 over the following three hours with codexmb review, all announced in the room. At 08:48Z hermes posted "All fixes implemented, built, tested, and committed on local main" for the same issues (#164 first, then the rest), on a machine with no GitHub credentials. Same pattern as the duplicate review sweep at 22:08Z the previous day: hermes picks up any task it can see regardless of who it was addressed to and of what has already been posted since. Two gates would have prevented both: (a) an explicit-addressee check before task pickup (this issue's item 3), (b) a "state of play" line listing open PRs per issue in the poller feed (items 1-2), so a late reader sees the work is done.
Merge pull request #93 from ThinkOffApp/fix/one-wake-path
ThinkOffApp commentedon Sep 7, 2026
Evidence for the "one wake path" item, 7 Sep 2026 (MacBook, @codexmb):
Two review requests, each posted ONCE with a single @codexmb mention, each answered TWICE by codexmb with differently worded LGTMs:
Receiver side is clean: /tmp/codex-webhook-wake.log shows one POST per message (the doubled lines are the receiver's own append + stdout redirected into the same file, a logging artifact, not two events) and one
wake fired. The poller heartbeat /tmp/iak-poller.heartbeat was fresh throughout (com.thinkoff.codexmb.poller running). So codex gets prompted twice per mention: once by the poller-delivered message and once by the webhook nudge ("check rooms [codex]"), and answers both.The Sep 2 gate (nudge refuses to type while the heartbeat is stale) covers the poller-down case only. Suggested inverse gate for the poller-up case: when the heartbeat is fresh (< 2 poll intervals), the webhook nudge should not type at all, or the poller should mark the message id as delivered so the nudge turn finds nothing new. Same pattern is likely on any agent that runs both a poller and a webhook nudge.
Logging nit while here: webhook-wake.mjs
log()both appends to WEBHOOK_WAKE_LOG and writes stdout; the supervisor redirects stdout to the same file, so every line is doubled in /tmp/codex-webhook-wake.log.chatgpt-codex-connector commentedon Sep 7, 2026
To use Codex here, create an environment for this repo.