diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d0e2383..884d332e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `brigade work import add/list/show/promote` to manage a gitignored local import inbox for scanner-discovered candidate work. - `brigade work import validate` and `brigade work import ingest` for scanner-authored JSONL import files. - `brigade work import memory-care` to convert `memory/cards/decay/refresh-queue.json` into local work imports. +- `brigade work import chat-sweep` to convert `.brigade/chat-memory-sweeps/latest.json` issues into local work imports. - `brigade work import triage` to group pending imports by source and kind. - `brigade work import dismiss` to close noisy imports without promoting them. - `brigade work import promote --all` with optional `--source` and `--kind` filters for batch promotion. @@ -70,6 +71,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `brigade work note` to append timestamped checkpoints to the active work session without ending it. - `brigade work doctor` to check dogfood config, Codex availability, local artifact paths, handoff inbox, ignore coverage, and latest run context for the daily work loop. - Workspace installs now include `.brigade/memory-care.example.json` as a scanner wiring contract for memory-care decay output. +- Workspace installs now include `.brigade/chat-memory-sweep.example.json` plus an OpenClaw memory-sweep cron fragment for nightly chat/session sweep wiring. - Roster-level and per-agent `timeout_seconds` controls for bounded CLI calls. - `brigade run --read-only` prompt policy for planning and review runs that should inspect and recommend only, with native `codex exec --sandbox read-only` enforcement for Codex agents. diff --git a/README.md b/README.md index cabfbaca..76c20ac0 100644 --- a/README.md +++ b/README.md @@ -273,6 +273,7 @@ Import inbox commands: - `brigade work import validate imports.jsonl` checks scanner output against [`docs/import-schema.md`](docs/import-schema.md). - `brigade work import ingest imports.jsonl` ingests scanner output. - `brigade work import memory-care` converts `memory/cards/decay/refresh-queue.json` into imports. +- `brigade work import chat-sweep` converts `.brigade/chat-memory-sweeps/latest.json` issues into imports. - `brigade work import triage` groups pending imports by source and kind. - `brigade work import show ` inspects one import. - `brigade work import dismiss ` removes noise. @@ -313,6 +314,7 @@ It also checks: - corrupt scan or refresh-queue JSON once the loop is wired Workspace installs include `.brigade/memory-care.example.json` as a scanner wiring contract for whatever scheduler or memory owner produces the decay files. +They also include `.brigade/chat-memory-sweep.example.json` for nightly chat/session sweep summaries. Missing memory-care state is advisory for fresh installs. Bootstrap truncation is a hard failure to prevent, not a cosmetic warning. diff --git a/ROADMAP.md b/ROADMAP.md index 5c126b71..bd890a0e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -35,11 +35,24 @@ Goal: make Brigade a safe target for local automations that discover useful work Goal: support the common places agent work happens without making any one chat product mandatory. -- Build adapters for Discord, Slack, Telegram, and export-based chat archives as separate scanner layers. +- Build adapters for Discord, Slack, ClickClack, Telegram, and export-based chat archives as separate scanner layers. - Convert surface-specific events into the local import inbox instead of writing memory directly. - Summarize private chat evidence, do not quote raw third-party messages into public docs or handoffs. - Use promotion gates so only reviewed, durable, or actionable items become tasks or memory handoffs. - Keep source metadata such as workspace, channel, thread, message range, and confidence local unless explicitly exported. +- Maintain a local provider registry for OpenClaw, Peter S, Vincent, and other chat plugins instead of hardcoding one product list. Seeded channel families include Discord, Slack, ClickClack, Telegram, WhatsApp, Signal, iMessage, BlueBubbles, Google Chat, Microsoft Teams, Matrix, Mattermost, Nextcloud Talk, Feishu, Line, QQ bot, Zalo, Nostr, IRC, Twitch, Tlon, Google Meet, voice-call transcripts, webhooks, and QA channels. +- Import nightly memory sweep `issues` into Brigade with `brigade work import chat-sweep`. Status: started with `.brigade/chat-memory-sweep.example.json`, OpenClaw cron fragment, and a command that converts sweep issues into local work imports. +- Add scheduler rules that spread memory ingest, crawler repair, chat sweeps, and OpenClaw updater jobs around update windows so upgrades do not race plugin or extension loads. + +## Later Phase: Backup And Recovery Visibility + +Goal: make backup health part of the same daily operator loop as chat, memory, and work imports. + +- Track restic backups to both NAS and cloud destinations. +- Surface latest NAS snapshot age, latest cloud snapshot age, prune result, `restic check` result, and restore rehearsal date. +- Send compact private backup summaries to the operator chat/status surface, including Discord or ClickClack when configured. +- Route stale snapshot, failed check, failed prune, missing mount, and restore-rehearsal overdue signals into `brigade work import` as incidents. +- Keep real hostnames, remote names, mount paths, webhook URLs, channel ids, and backup passwords out of public templates. ## Later Phase: Cybersecurity Plugin diff --git a/docs/import-schema.md b/docs/import-schema.md index 20e2cbb9..72ba6681 100644 --- a/docs/import-schema.md +++ b/docs/import-schema.md @@ -10,11 +10,12 @@ Imports are local review items. They do not write canonical memory directly. brigade work import validate imports.jsonl brigade work import ingest imports.jsonl brigade work import memory-care +brigade work import chat-sweep brigade work import triage brigade work import promote --all --source memory-care --kind task ``` -`validate` checks a JSONL file without writing. `ingest` appends valid records into `.brigade/work/imports/inbox.jsonl`, skipping duplicate pending records with the same source, kind, and normalized text. `memory-care` reads `memory/cards/decay/refresh-queue.json` and converts queued cards into task imports. +`validate` checks a JSONL file without writing. `ingest` appends valid records into `.brigade/work/imports/inbox.jsonl`, skipping duplicate pending records with the same source, kind, and normalized text. `memory-care` reads `memory/cards/decay/refresh-queue.json` and converts queued cards into task imports. `chat-sweep` reads `.brigade/chat-memory-sweeps/latest.json` and converts sweep `issues` into incident-oriented imports. ## Record Shape @@ -38,6 +39,9 @@ Recommended metadata keys: - `card_file`: memory card path for memory-care records. - `reason`: short reason the item was produced. +- `sweep_path`: local chat memory sweep JSON path. +- `issue_title`: title from a chat memory sweep issue. +- `issue_source`: original issue source such as cron, delivery, crawler, or bridge. - `workspace`: chat workspace or source workspace. - `channel`: chat channel or surface name. - `thread`: thread id, message range, or export locator. @@ -79,3 +83,46 @@ brigade work import triage ``` The producer writes `task` imports with source `memory-care`, preserving `card_file`, `reason`, and `queue_path` metadata. + +## Chat Memory Sweep Producer + +The chat-sweep producer reads this summary by default: + +```text +.brigade/chat-memory-sweeps/latest.json +``` + +Minimal sweep shape: + +```json +{ + "generated_at": "2026-05-26T22:09:00-04:00", + "sessions": { + "listed": 24, + "reviewed": 10, + "durable": 1 + }, + "issues": [ + { + "title": "Cron delivery failure", + "summary": "Recent message delivery failed.", + "kind": "incident", + "source": "cron", + "severity": "warning", + "metadata": { + "surface": "discord", + "local_locator": "crawler://discord/example" + } + } + ] +} +``` + +Run: + +```bash +brigade work import chat-sweep +brigade work import triage +``` + +The producer writes imports with source `chat-memory-sweep`, preserving local locators and summary metadata. It does not import raw transcripts, persisted memory entries, or skipped buckets. diff --git a/src/brigade/cli.py b/src/brigade/cli.py index 8bbdfa7d..20ea239e 100644 --- a/src/brigade/cli.py +++ b/src/brigade/cli.py @@ -198,6 +198,17 @@ def _build_parser() -> argparse.ArgumentParser: ) p_work_import_memory_care.add_argument("--dry-run", action="store_true", help="Report without writing imports.") p_work_import_memory_care.add_argument("--json", action="store_true", help="Print machine-readable JSON.") + p_work_import_chat_sweep = import_sub.add_parser("chat-sweep", help="Import chat memory sweep issues.") + p_work_import_chat_sweep.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to update.") + p_work_import_chat_sweep.add_argument( + "--input", + dest="input_path", + type=Path, + default=None, + help="Chat memory sweep JSON. Defaults to .brigade/chat-memory-sweeps/latest.json under target.", + ) + p_work_import_chat_sweep.add_argument("--dry-run", action="store_true", help="Report without writing imports.") + p_work_import_chat_sweep.add_argument("--json", action="store_true", help="Print machine-readable JSON.") p_work_import_triage = import_sub.add_parser("triage", help="Group pending imports by source and kind.") p_work_import_triage.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to inspect.") p_work_import_triage.add_argument("--json", action="store_true", help="Print machine-readable JSON.") @@ -684,6 +695,13 @@ def main(argv=None) -> int: dry_run=args.dry_run, json_output=args.json, ) + if args.import_command == "chat-sweep": + return work_cmd.import_chat_sweep( + target=args.target, + input_path=args.input_path, + dry_run=args.dry_run, + json_output=args.json, + ) if args.import_command == "triage": return work_cmd.import_triage(target=args.target, json_output=args.json, limit=args.limit) if args.import_command == "show": diff --git a/src/brigade/install.py b/src/brigade/install.py index cdd274d5..e732239c 100644 --- a/src/brigade/install.py +++ b/src/brigade/install.py @@ -67,6 +67,7 @@ def build_gitignore_block(selection: Selection) -> str: ".brigade/runs/", ".brigade/scrub-cache/", ".brigade/security/", + ".brigade/chat-memory-sweeps/", ".brigade/work/", GITIGNORE_END, "", diff --git a/src/brigade/templates/depth/workspace.json b/src/brigade/templates/depth/workspace.json index 6a642b44..6bf55422 100644 --- a/src/brigade/templates/depth/workspace.json +++ b/src/brigade/templates/depth/workspace.json @@ -21,6 +21,7 @@ {"src": "memory/cards/obsidian-notes.md", "dst": "memory/cards/obsidian-notes.md"}, {"src": "memory/cards/backup-restic.md", "dst": "memory/cards/backup-restic.md"}, {"src": "memory/memory-care.example.json", "dst": ".brigade/memory-care.example.json"}, + {"src": "memory/chat-memory-sweep.example.json", "dst": ".brigade/chat-memory-sweep.example.json"}, {"src": "skills/note/SKILL.md", "dst": "skills/note/SKILL.md"}, {"src": "scripts/backup-restic.sh", "dst": "scripts/backup-restic.sh", "mode": "0755"} ], diff --git a/src/brigade/templates/harnesses/openclaw.json b/src/brigade/templates/harnesses/openclaw.json index e5e538c7..19558e28 100644 --- a/src/brigade/templates/harnesses/openclaw.json +++ b/src/brigade/templates/harnesses/openclaw.json @@ -6,6 +6,7 @@ {"src": "openclaw/model-aliases.openclaw.json", "dst": ".brigade/openclaw/model-aliases.openclaw.json"}, {"src": "openclaw/ollama-memory-search.openclaw.json", "dst": ".brigade/openclaw/ollama-memory-search.openclaw.json"}, {"src": "openclaw/acp-escalation.openclaw.json", "dst": ".brigade/openclaw/acp-escalation.openclaw.json"}, + {"src": "openclaw/memory-sweep-cron.openclaw.json", "dst": ".brigade/openclaw/memory-sweep-cron.openclaw.json"}, {"src": "openclaw/README.md", "dst": ".brigade/openclaw/README.md"} ], "dirs": [], diff --git a/src/brigade/templates/memory/cards/backup-restic.md b/src/brigade/templates/memory/cards/backup-restic.md index 7a342a3d..e132549d 100644 --- a/src/brigade/templates/memory/cards/backup-restic.md +++ b/src/brigade/templates/memory/cards/backup-restic.md @@ -117,6 +117,16 @@ Log file path: Worth wiring into the morning report (`memory/cards/pipeline-standups.md`): grep recent logs for `ERROR` and surface in the briefing. A silent backup that has been failing for three weeks is the second-worst kind of bug. +If chat is part of your operations loop, send backup summaries to the same private status surface you already use for agent reports. Keep the status compact: + +- latest NAS snapshot age +- latest cloud snapshot age +- most recent `restic check` result +- prune result +- restore rehearsal date + +Route stale or failed backup checks into the Brigade work inbox as incidents. Do not publish real repo names, hostnames, webhook URLs, cloud remote names, or channel ids in public docs. + ## Anti-patterns - **One destination only.** Single point of failure. diff --git a/src/brigade/templates/memory/cards/chat-surface-crawlers.md b/src/brigade/templates/memory/cards/chat-surface-crawlers.md index e824dfa2..788bf1a5 100644 --- a/src/brigade/templates/memory/cards/chat-surface-crawlers.md +++ b/src/brigade/templates/memory/cards/chat-surface-crawlers.md @@ -41,15 +41,23 @@ Each crawler is platform-specific (auth differs, intents differ, rate limits dif | Platform | Crawler (suggested name) | Status | |----------|--------------------------|--------| | Discord | [`discrawl`](https://github.com/solomonneas/discrawl) | Available. Bot-token, SQLite + FTS5, git-snapshot read mode. | +| ClickClack | `clickclack` | Planned adapter. Use its agent-friendly CLI, export command, realtime event API, incoming webhook, and slash command surfaces. | | Slack | `slackcrawl` | Pattern-name. Bot-token + RTM events; same SQLite shape. | | WhatsApp | `whatsappcrawl` | Pattern-name. Likely via WhatsApp Business API or Multi-Device session bridge. | | Telegram | `tgcrawl` | Pattern-name. Bot API + MTProto for history backfill. | | iMessage | `imescrawl` | Pattern-name. macOS-only; reads from local `chat.db`. | | Signal | `signalcrawl` | Pattern-name. Signal-cli session export. | +| Matrix | `matrixcrawl` | Pattern-name. Room archive plus event sync. | +| Mattermost | `mattermostcrawl` | Pattern-name. Channel and thread export shape close to Slack. | +| Google Chat / Teams / Feishu | product-specific crawlers | Pattern-name. Enterprise chat adapters should preserve workspace, channel, thread, and user locators. | | Email (Gmail/IMAP) | `mailcrawl` | Pattern-name. IMAP IDLE for tail, full-folder sync for repair. | The names above follow the `crawl` convention used by `discrawl`. Swap them for whatever tool you actually use - the contract matters, not the binary name. +Known channel families worth scanning when present: Discord, Slack, ClickClack, Telegram, WhatsApp, Signal, iMessage, BlueBubbles, Google Chat, Microsoft Teams, Matrix, Mattermost, Nextcloud Talk, Feishu, Line, QQ bot, Zalo, Nostr, IRC, Twitch, Tlon, Google Meet, voice-call transcripts, incoming webhooks, and QA channels. + +Do not hardcode this list into business logic. Treat it as a registry that local installs can extend as new OpenClaw, Peter S, Vincent, or other chat plugins appear. + ## Discrawl reference (the tested one) `discrawl` is the canonical implementation: @@ -83,6 +91,15 @@ discrawl query --since "$since" --channel "#decisions" --format json | The scanner writes one Memory Handoff per durable finding. The conservative ingester routes those handoffs into cards / runbooks / learnings the same way it routes anything else. +For operational failures found during the same pass, write a compact sweep file and import the issues into the daily inbox: + +```bash +brigade work import chat-sweep --input .brigade/chat-memory-sweeps/latest.json +brigade work import triage +``` + +This makes delivery failures, stale bridge jobs, and crawler gaps visible without promoting raw chat content. + ## Privacy boundary Chat archives are *intimate*. The crawler's local SQLite often contains: diff --git a/src/brigade/templates/memory/cards/memory-scanner.md b/src/brigade/templates/memory/cards/memory-scanner.md index 44cafbdf..eb798d55 100644 --- a/src/brigade/templates/memory/cards/memory-scanner.md +++ b/src/brigade/templates/memory/cards/memory-scanner.md @@ -38,7 +38,7 @@ A real implementation (typical nightly cadence): 1. **List recent sessions.** Last 12-24 hours, across all harnesses connected to canonical memory. 2. **Skip noise.** Cron-spawned sessions, heartbeat/reminder-only sessions, empty subagent shells, pure delivery mirror / announce-only sessions. -3. **Prioritize real human-facing sessions first.** Discord, WhatsApp, Telegram, Slack, manual ACP threads. These are where decisions, corrections, and preferences actually land. +3. **Prioritize real human-facing sessions first.** Discord, ClickClack, WhatsApp, Telegram, Slack, manual ACP threads. These are where decisions, corrections, and preferences actually land. 4. **Start with summaries.** Only fetch deeper history for sessions that clearly contain durable decisions, corrections, preferences, project changes, new tooling facts, or new published outputs. 5. **Cap deep review** to the top N most promising sessions (typical: 8) unless there is an obvious reason to exceed it. 6. **Avoid duplication.** Do not re-promote facts that already exist as cards or in daily-log entries. @@ -62,22 +62,41 @@ A real implementation (typical nightly cadence): The scanner reports back on what it did: ```text -Sessions listed: N -Sessions deeply reviewed: N -Sessions with meaningful content: N -Persisted: [bullets] -Skipped: [short bullets] -Net result: [1-3 bullets] +Memory Sweep - YYYY-MM-DD + +Sessions: N listed | N reviewed | N with durable content + +Persisted: +- card-or-file.md - updated with concise durable change. + +Issues: +- Short operational failure or delivery problem that needs review. + +Skipped: +- N cron/agent sessions - routine, heartbeat-only, no-reply, or operational noise after signal pass. +- N chat/manual sessions - delivery-only, output artifacts, or no durable preference/tooling/project change. + +Net result: Captured durable context and surfaced issues that should not be treated as noise. ``` This gives you an audit trail and a heartbeat for the scanner itself. If "persisted" is empty for a week, either nothing durable happened or the scanner stopped firing. +The scanner should also write a local machine-readable summary at: + +```text +.brigade/chat-memory-sweeps/latest.json +``` + +Use `.brigade/chat-memory-sweep.example.json` as the contract. `brigade work import chat-sweep` imports only the `issues` array into the local work inbox. Raw transcripts, private message bodies, and channel exports stay in crawler archives. + ## Scheduling Common cadence (matches reference cookbook): nightly at quiet hours, after [pipeline-standups](pipeline-standups.md) have run and the day's activity has settled. Avoid promoting during active sessions because card writes invalidate prefix caches. +Spread recurring jobs around update windows. Do not anchor memory ingest, chat sweep, crawler repair, and updater jobs on the same minute or the same five-minute window. Prefer staggered minute offsets such as `:15` and `:45` for frequent ingest jobs when an updater runs near the top of the hour. + ## Implementation surface `brigade` ships the contract; it does not ship the scanner agent itself. Wire it as: @@ -85,6 +104,7 @@ Avoid promoting during active sessions because card writes invalidate prefix cac - a cron job that spawns an isolated agent session with a "review last 12h and persist durable facts" prompt - the prompt should embed the skip-rules and cost controls above - output goes either directly to `memory/cards/*.md` (if your harness can write there) or through `.claude/memory-handoffs/` for the conservative ingester to route +- issue summaries also go to `.brigade/chat-memory-sweeps/latest.json` so the daily work loop can review operational failures ## Relationship to Memory Care diff --git a/src/brigade/templates/memory/chat-memory-sweep.example.json b/src/brigade/templates/memory/chat-memory-sweep.example.json new file mode 100644 index 00000000..0cfe3874 --- /dev/null +++ b/src/brigade/templates/memory/chat-memory-sweep.example.json @@ -0,0 +1,131 @@ +{ + "_brigade_version": "0.1.0", + "_description": "Example output contract for a nightly chat/session memory sweep. Keep raw chat transcripts in crawler archives; this file contains summaries and local source locators only.", + "generated_at": "2026-05-26T22:09:00-04:00", + "window": { + "since": "2026-05-25T22:00:00-04:00", + "until": "2026-05-26T22:00:00-04:00" + }, + "sessions": { + "listed": 24, + "reviewed": 10, + "durable": 1 + }, + "surfaces": [ + { + "id": "discord", + "crawler": "discrawl", + "status": "enabled", + "query": "discrawl query --since --format json" + }, + { + "id": "slack", + "crawler": "slackcrawl", + "status": "planned", + "query": "slackcrawl query --since --format json" + }, + { + "id": "clickclack", + "crawler": "clickclack", + "status": "planned", + "query": "clickclack export --out .brigade/chat-memory-sweeps/clickclack-export.json" + }, + { + "id": "telegram", + "crawler": "tgcrawl", + "status": "planned", + "query": "tgcrawl query --since --format json" + }, + { + "id": "matrix", + "crawler": "matrixcrawl", + "status": "planned", + "query": "matrixcrawl query --since --format json" + }, + { + "id": "mattermost", + "crawler": "mattermostcrawl", + "status": "planned", + "query": "mattermostcrawl query --since --format json" + }, + { + "id": "signal", + "crawler": "signalcrawl", + "status": "planned", + "query": "signalcrawl query --since --format json" + }, + { + "id": "imessage", + "crawler": "imescrawl", + "status": "planned", + "query": "imescrawl query --since --format json" + }, + { + "id": "webhooks", + "crawler": "webhook-archive", + "status": "planned", + "query": "webhook-archive query --since --format json" + } + ], + "known_channel_families": [ + "discord", + "slack", + "clickclack", + "telegram", + "whatsapp", + "signal", + "imessage", + "bluebubbles", + "googlechat", + "msteams", + "matrix", + "mattermost", + "nextcloud-talk", + "feishu", + "line", + "qqbot", + "zalo", + "nostr", + "irc", + "twitch", + "tlon", + "google-meet", + "voice-call", + "webhooks", + "qa-channel" + ], + "persisted": [ + { + "target": "memory/cards/example-project-state.md", + "action": "updated", + "summary": "Updated durable project state from reviewed chat and agent sessions." + } + ], + "issues": [ + { + "title": "Example cron delivery failure", + "summary": "Recent delivery failure surfaced during the nightly sweep.", + "kind": "incident", + "source": "cron", + "severity": "warning", + "metadata": { + "local_locator": "crawler://example/session-or-thread-id", + "surface": "discord", + "raw_transcript": "not included" + } + } + ], + "skipped": [ + { + "count": 14, + "label": "cron/agent sessions", + "reason": "routine, heartbeat-only, no-reply, or operational noise after signal pass" + }, + { + "count": 8, + "label": "chat/manual sessions", + "reason": "delivery-only, output artifacts, or no durable preference/tooling/project change" + } + ], + "net_result": "Captured one durable update and surfaced one operational issue for review." +} diff --git a/src/brigade/templates/openclaw/README.md b/src/brigade/templates/openclaw/README.md index 80787b81..6b19c82b 100644 --- a/src/brigade/templates/openclaw/README.md +++ b/src/brigade/templates/openclaw/README.md @@ -9,6 +9,7 @@ These are JSON fragments meant to be inspected and merged by hand into your `~/. | `model-aliases.openclaw.json` | Suggested alias map under `agents.defaults.models`. | | `ollama-memory-search.openclaw.json` | Local Ollama embeddings for memory search. | | `acp-escalation.openclaw.json` | ACP escalation lane via the `acpx` plugin. | +| `memory-sweep-cron.openclaw.json` | Nightly memory sweep cron stub plus chat-sweep JSON output guidance. | ## Merge @@ -38,3 +39,4 @@ The doctor reports which fragments your live config has picked up and which chec - Aliases referencing `` placeholders must be replaced with real ids before merging. - The ACP fragment assumes you have already installed `acpx` (see [solos-cookbook/ai-stack/acp-claude-code.md](https://github.com/solomonneas/solos-cookbook) for the install path). - `openclaw doctor` (the OpenClaw tool, not `brigade doctor`) has historically rewritten `openai-codex/*` prefixes on certain versions. If you use OAuth-only auth, audit `agents.defaults.model.primary` after any OpenClaw upgrade. +- Stagger ingest and sweep cron minutes around OpenClaw update windows. Avoid running memory ingest at the same time as updater jobs. diff --git a/src/brigade/templates/openclaw/memory-sweep-cron.openclaw.json b/src/brigade/templates/openclaw/memory-sweep-cron.openclaw.json new file mode 100644 index 00000000..159d4cec --- /dev/null +++ b/src/brigade/templates/openclaw/memory-sweep-cron.openclaw.json @@ -0,0 +1,40 @@ +{ + "_comment": [ + "FRAGMENT - inspect before merging into your OpenClaw cron jobs.", + "Generated by `brigade openclaw-fragments`. Brigade never mutates your live config.", + "", + "Reference shape from the Brigade dogfood setup:", + "- nightly Memory Sweep - Session Review at 22:00 local time", + "- separate handoff ingest on staggered minute anchors such as :15 and :45", + "- separate crawler cleanup/sync jobs for surfaces such as discrawl", + "- avoid scheduling memory ingest jobs inside OpenClaw update windows", + "", + "Replace placeholders before use. Do not include raw chat transcripts in the sweep output." + ], + "jobs": [ + { + "name": "Memory Sweep - Session Review", + "enabled": true, + "agentId": "main", + "schedule": { + "kind": "cron", + "expr": "0 22 * * *", + "tz": "" + }, + "sessionTarget": "", + "wakeMode": "new-session", + "payload": { + "kind": "message", + "model": "cron", + "thinking": "medium", + "timeoutSeconds": 1800, + "message": "Review the last 24 hours of agent sessions and connected chat crawler archives. Produce a Memory Sweep summary with counts, persisted memory updates, operational issues, skipped buckets, and net result. Summarize only. Do not quote raw chat messages. Write durable findings through the configured memory handoff path or canonical memory owner. Also write a local sweep JSON compatible with .brigade/chat-memory-sweep.example.json so Brigade can import Issues into the work inbox." + }, + "delivery": { + "mode": "announce", + "channel": "", + "to": "" + } + } + ] +} diff --git a/src/brigade/templates/workspace/AGENTS.md b/src/brigade/templates/workspace/AGENTS.md index 008b20ef..9cc6f6ad 100644 --- a/src/brigade/templates/workspace/AGENTS.md +++ b/src/brigade/templates/workspace/AGENTS.md @@ -123,6 +123,8 @@ A typical day has two short cross-harness summaries plus a session-review pass a Standups summarize state already in memory. The memory scanner promotes durable findings *into* memory from the day's sessions. Memory care checks existing cards for stale facts. Run order: standup -> scanner -> overnight ingester sweeps -> memory-care scan -> morning report next day. +Stagger frequent ingest jobs around updater windows. Avoid putting memory ingest, chat sweeps, crawler repair, and OpenClaw updates on the same minute. + ## Multi-Agent Workflow Configure your agent roster in the table below. The default shape: diff --git a/src/brigade/templates/workspace/TOOLS.md b/src/brigade/templates/workspace/TOOLS.md index 6038f660..0422beb0 100644 --- a/src/brigade/templates/workspace/TOOLS.md +++ b/src/brigade/templates/workspace/TOOLS.md @@ -63,10 +63,23 @@ discrawl tail # live event stream discrawl search '' # local FTS # Other surfaces follow the same shape (slackcrawl, tgcrawl, whatsappcrawl, mailcrawl) + +# ClickClack +clickclack send --channel general "status" +clickclack messages list --channel general +clickclack export --out .brigade/chat-memory-sweeps/clickclack-export.json + +# Sweep output +test -f .brigade/chat-memory-sweeps/latest.json +jq '.sessions, .issues' .brigade/chat-memory-sweeps/latest.json +brigade work import chat-sweep --input .brigade/chat-memory-sweeps/latest.json +brigade work import triage ``` See `memory/cards/chat-surface-crawlers.md` for the full pattern. +Stagger crawler repair, memory ingest, and updater cron jobs. Avoid placing memory ingest jobs inside known OpenClaw update windows. + ## Publish Guard ```bash diff --git a/src/brigade/work_cmd.py b/src/brigade/work_cmd.py index 95a30ea0..46dbe20a 100644 --- a/src/brigade/work_cmd.py +++ b/src/brigade/work_cmd.py @@ -1759,6 +1759,118 @@ def import_memory_care( return 0 +def import_chat_sweep( + *, + target: Path, + input_path: Path | None = None, + dry_run: bool = False, + json_output: bool = False, +) -> int: + target = target.expanduser().resolve() + if not target.is_dir(): + print(f"error: --target is not a directory: {target}", file=sys.stderr) + return 2 + sweep_path = ( + input_path.expanduser().resolve() + if input_path is not None + else target / ".brigade" / "chat-memory-sweeps" / "latest.json" + ) + if not sweep_path.is_file(): + print(f"error: chat memory sweep not found: {sweep_path}", file=sys.stderr) + return 2 + try: + payload = json.loads(sweep_path.read_text()) + except json.JSONDecodeError as exc: + print(f"error: invalid chat memory sweep JSON: {exc}", file=sys.stderr) + return 2 + if not isinstance(payload, dict): + print(f"error: chat memory sweep must be an object: {sweep_path}", file=sys.stderr) + return 2 + issues = payload.get("issues", []) + if not isinstance(issues, list): + print(f"error: chat memory sweep `issues` must be a list: {sweep_path}", file=sys.stderr) + return 2 + + records: list[dict[str, Any]] = [] + generated_at = payload.get("generated_at") + for index, issue in enumerate(issues, start=1): + if not isinstance(issue, dict): + print(f"error: chat memory sweep issue {index} must be an object", file=sys.stderr) + return 2 + title = issue.get("title") + if not isinstance(title, str) or not title.strip(): + print(f"error: chat memory sweep issue {index} requires title", file=sys.stderr) + return 2 + kind = issue.get("kind", "incident") + if not isinstance(kind, str) or kind not in IMPORT_KINDS: + print(f"error: chat memory sweep issue {index} kind must be one of: {', '.join(IMPORT_KINDS)}", file=sys.stderr) + return 2 + metadata = issue.get("metadata", {}) + if metadata is None: + metadata = {} + if not isinstance(metadata, dict): + print(f"error: chat memory sweep issue {index} metadata must be an object", file=sys.stderr) + return 2 + + summary = issue.get("summary") + summary_text = summary.strip() if isinstance(summary, str) and summary.strip() else "" + severity = issue.get("severity") + severity_text = severity.strip() if isinstance(severity, str) and severity.strip() else "" + issue_source = issue.get("source") + issue_source_text = issue_source.strip() if isinstance(issue_source, str) and issue_source.strip() else "" + rendered_title = title.strip() + severity_prefix = f" [{severity_text}]" if severity_text else "" + text = f"Review memory sweep issue{severity_prefix} {rendered_title}" + if summary_text: + text = f"{text}: {summary_text}" + + record_metadata = dict(metadata) + record_metadata.update( + { + "sweep_path": str(sweep_path), + "issue_title": rendered_title, + } + ) + if issue_source_text: + record_metadata["issue_source"] = issue_source_text + if severity_text: + record_metadata["severity"] = severity_text + if isinstance(generated_at, str) and generated_at.strip(): + record_metadata["generated_at"] = generated_at.strip() + + records.append( + { + "text": text, + "kind": kind, + "source": "chat-memory-sweep", + "metadata": record_metadata, + } + ) + + imported, skipped = _append_import_records(target, records, dry_run=dry_run) + output = { + "input": str(sweep_path), + "imports_path": str(_imports_path(target)), + "dry_run": dry_run, + "issues": len(issues), + "imported": len(imported), + "skipped_duplicates": len(skipped), + "imports": imported, + } + if json_output: + print(json.dumps(output, indent=2, sort_keys=True)) + return 0 + print(f"chat memory sweep: {sweep_path}") + print(f"imports_path: {_imports_path(target)}") + print(f"dry_run: {dry_run}") + print(f"issues: {len(issues)}") + print(f"imported: {len(imported)}") + print(f"skipped_duplicates: {len(skipped)}") + for item in imported: + print(f"- {item.get('id')} [{item.get('kind')}] {_short(str(item.get('text', '')))}") + return 0 + + def import_triage(*, target: Path, json_output: bool = False, limit: int = 50) -> int: if limit < 1: print("error: --limit must be a positive integer", file=sys.stderr) diff --git a/tests/test_gitignore.py b/tests/test_gitignore.py index 82475d84..95ffd74e 100644 --- a/tests/test_gitignore.py +++ b/tests/test_gitignore.py @@ -29,6 +29,7 @@ def test_init_creates_gitignore_when_missing(tmp_target: Path): assert ".brigade/security.toml" in gi assert ".brigade/runs/" in gi assert ".brigade/security/" in gi + assert ".brigade/chat-memory-sweeps/" in gi assert ".brigade/work/" in gi diff --git a/tests/test_init.py b/tests/test_init.py index e0c05586..c617ebae 100644 --- a/tests/test_init.py +++ b/tests/test_init.py @@ -93,6 +93,7 @@ def test_workspace_install_includes_memory_cards(tmp_target: Path): assert (tmp_target / "memory" / "handoff-inbox").is_dir() assert (tmp_target / ".claude" / "memory-handoffs" / "processed").is_dir() assert (tmp_target / ".brigade" / "memory-care.example.json").is_file() + assert (tmp_target / ".brigade" / "chat-memory-sweep.example.json").is_file() # skill + script land at the right paths, executable bit on the script assert (tmp_target / "skills" / "note" / "SKILL.md").is_file() backup = tmp_target / "scripts" / "backup-restic.sh" @@ -110,6 +111,7 @@ def test_openclaw_install_extends_workspace(tmp_target: Path): assert (fragments_dir / "model-aliases.openclaw.json").is_file() assert (fragments_dir / "ollama-memory-search.openclaw.json").is_file() assert (fragments_dir / "acp-escalation.openclaw.json").is_file() + assert (fragments_dir / "memory-sweep-cron.openclaw.json").is_file() def test_hermes_install_writes_experimental_fragments(tmp_target: Path): diff --git a/tests/test_work_cmd.py b/tests/test_work_cmd.py index 47610d6a..72be626b 100644 --- a/tests/test_work_cmd.py +++ b/tests/test_work_cmd.py @@ -804,6 +804,58 @@ def test_work_import_memory_care_reads_refresh_queue(tmp_path, monkeypatch, caps assert item["metadata"]["reason"] == "source-of-truth changed" +def test_work_import_chat_sweep_reads_issues(tmp_path, monkeypatch, capsys): + _init_git_repo(tmp_path) + monkeypatch.setattr( + work_cmd, + "_now", + lambda: datetime(2026, 5, 26, 12, 0, 0, tzinfo=timezone.utc), + ) + sweep = tmp_path / ".brigade" / "chat-memory-sweeps" / "latest.json" + sweep.parent.mkdir(parents=True) + _write_json( + sweep, + { + "generated_at": "2026-05-26T22:09:00-04:00", + "sessions": {"listed": 24, "reviewed": 10, "durable": 1}, + "issues": [ + { + "title": "Cron delivery failure", + "summary": "Recent message delivery failed.", + "kind": "incident", + "source": "cron", + "severity": "warning", + "metadata": { + "surface": "discord", + "local_locator": "crawler://discord/example", + }, + } + ], + }, + ) + + assert work_cmd.import_chat_sweep(target=tmp_path) == 0 + out = capsys.readouterr().out + assert f"chat memory sweep: {sweep}" in out + assert "issues: 1" in out + assert "imported: 1" in out + assert work_cmd.import_chat_sweep(target=tmp_path) == 0 + out = capsys.readouterr().out + assert "imported: 0" in out + assert "skipped_duplicates: 1" in out + + assert work_cmd.import_list(target=tmp_path, json_output=True) == 0 + payload = json.loads(capsys.readouterr().out) + item = payload["imports"][0] + assert item["kind"] == "incident" + assert item["source"] == "chat-memory-sweep" + assert item["text"] == "Review memory sweep issue [warning] Cron delivery failure: Recent message delivery failed." + assert item["metadata"]["surface"] == "discord" + assert item["metadata"]["issue_source"] == "cron" + assert item["metadata"]["severity"] == "warning" + assert item["metadata"]["sweep_path"] == str(sweep) + + def test_work_import_triage_groups_pending_imports(tmp_path, monkeypatch, capsys): _init_git_repo(tmp_path) monkeypatch.setattr( @@ -1421,6 +1473,10 @@ def fake_import_memory_care(**kwargs): seen.append(("memory-care", kwargs)) return 0 + def fake_import_chat_sweep(**kwargs): + seen.append(("chat-sweep", kwargs)) + return 0 + def fake_import_triage(**kwargs): seen.append(("triage", kwargs)) return 0 @@ -1442,6 +1498,7 @@ def fake_import_dismiss(**kwargs): monkeypatch.setattr(work_cmd, "import_validate", fake_import_validate) monkeypatch.setattr(work_cmd, "import_ingest", fake_import_ingest) monkeypatch.setattr(work_cmd, "import_memory_care", fake_import_memory_care) + monkeypatch.setattr(work_cmd, "import_chat_sweep", fake_import_chat_sweep) monkeypatch.setattr(work_cmd, "import_triage", fake_import_triage) monkeypatch.setattr(work_cmd, "import_show", fake_import_show) monkeypatch.setattr(work_cmd, "import_promote", fake_import_promote) @@ -1500,6 +1557,22 @@ def fake_import_dismiss(**kwargs): ) == 0 ) + assert ( + cli.main( + [ + "work", + "import", + "chat-sweep", + "--target", + str(tmp_path), + "--input", + str(tmp_path / "latest-sweep.json"), + "--dry-run", + "--json", + ] + ) + == 0 + ) assert cli.main(["work", "import", "triage", "--target", str(tmp_path), "--json", "--limit", "4"]) == 0 assert cli.main(["work", "import", "show", "imp123", "--target", str(tmp_path)]) == 0 assert ( @@ -1551,6 +1624,15 @@ def fake_import_dismiss(**kwargs): "json_output": True, }, ), + ( + "chat-sweep", + { + "target": tmp_path, + "input_path": tmp_path / "latest-sweep.json", + "dry_run": True, + "json_output": True, + }, + ), ("triage", {"target": tmp_path, "json_output": True, "limit": 4}), ("show", {"target": tmp_path, "import_id": "imp123"}), (