Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `brigade work import add/list/show/promote` to manage a gitignored local import inbox for scanner-discovered candidate work.
- `brigade work import validate` and `brigade work import ingest` for scanner-authored JSONL import files.
- `brigade work import memory-care` to convert `memory/cards/decay/refresh-queue.json` into local work imports.
- `brigade work import chat-sweep` to convert `.brigade/chat-memory-sweeps/latest.json` issues into local work imports.
- `brigade work import triage` to group pending imports by source and kind.
- `brigade work import dismiss` to close noisy imports without promoting them.
- `brigade work import promote --all` with optional `--source` and `--kind` filters for batch promotion.
Expand All @@ -70,6 +71,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `brigade work note` to append timestamped checkpoints to the active work session without ending it.
- `brigade work doctor` to check dogfood config, Codex availability, local artifact paths, handoff inbox, ignore coverage, and latest run context for the daily work loop.
- Workspace installs now include `.brigade/memory-care.example.json` as a scanner wiring contract for memory-care decay output.
- Workspace installs now include `.brigade/chat-memory-sweep.example.json` plus an OpenClaw memory-sweep cron fragment for nightly chat/session sweep wiring.
- Roster-level and per-agent `timeout_seconds` controls for bounded CLI calls.
- `brigade run --read-only` prompt policy for planning and review runs that should inspect and recommend only, with native `codex exec --sandbox read-only` enforcement for Codex agents.

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ Import inbox commands:
- `brigade work import validate imports.jsonl` checks scanner output against [`docs/import-schema.md`](docs/import-schema.md).
- `brigade work import ingest imports.jsonl` ingests scanner output.
- `brigade work import memory-care` converts `memory/cards/decay/refresh-queue.json` into imports.
- `brigade work import chat-sweep` converts `.brigade/chat-memory-sweeps/latest.json` issues into imports.
- `brigade work import triage` groups pending imports by source and kind.
- `brigade work import show <import-id>` inspects one import.
- `brigade work import dismiss <import-id>` removes noise.
Expand Down Expand Up @@ -313,6 +314,7 @@ It also checks:
- corrupt scan or refresh-queue JSON once the loop is wired

Workspace installs include `.brigade/memory-care.example.json` as a scanner wiring contract for whatever scheduler or memory owner produces the decay files.
They also include `.brigade/chat-memory-sweep.example.json` for nightly chat/session sweep summaries.
Missing memory-care state is advisory for fresh installs.
Bootstrap truncation is a hard failure to prevent, not a cosmetic warning.

Expand Down
15 changes: 14 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,24 @@ Goal: make Brigade a safe target for local automations that discover useful work

Goal: support the common places agent work happens without making any one chat product mandatory.

- Build adapters for Discord, Slack, Telegram, and export-based chat archives as separate scanner layers.
- Build adapters for Discord, Slack, ClickClack, Telegram, and export-based chat archives as separate scanner layers.
- Convert surface-specific events into the local import inbox instead of writing memory directly.
- Summarize private chat evidence, do not quote raw third-party messages into public docs or handoffs.
- Use promotion gates so only reviewed, durable, or actionable items become tasks or memory handoffs.
- Keep source metadata such as workspace, channel, thread, message range, and confidence local unless explicitly exported.
- Maintain a local provider registry for OpenClaw, Peter S, Vincent, and other chat plugins instead of hardcoding one product list. Seeded channel families include Discord, Slack, ClickClack, Telegram, WhatsApp, Signal, iMessage, BlueBubbles, Google Chat, Microsoft Teams, Matrix, Mattermost, Nextcloud Talk, Feishu, Line, QQ bot, Zalo, Nostr, IRC, Twitch, Tlon, Google Meet, voice-call transcripts, webhooks, and QA channels.
- Import nightly memory sweep `issues` into Brigade with `brigade work import chat-sweep`. Status: started with `.brigade/chat-memory-sweep.example.json`, OpenClaw cron fragment, and a command that converts sweep issues into local work imports.
- Add scheduler rules that spread memory ingest, crawler repair, chat sweeps, and OpenClaw updater jobs around update windows so upgrades do not race plugin or extension loads.

## Later Phase: Backup And Recovery Visibility

Goal: make backup health part of the same daily operator loop as chat, memory, and work imports.

- Track restic backups to both NAS and cloud destinations.
- Surface latest NAS snapshot age, latest cloud snapshot age, prune result, `restic check` result, and restore rehearsal date.
- Send compact private backup summaries to the operator chat/status surface, including Discord or ClickClack when configured.
- Route stale snapshot, failed check, failed prune, missing mount, and restore-rehearsal overdue signals into `brigade work import` as incidents.
- Keep real hostnames, remote names, mount paths, webhook URLs, channel ids, and backup passwords out of public templates.

## Later Phase: Cybersecurity Plugin

Expand Down
49 changes: 48 additions & 1 deletion docs/import-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@ Imports are local review items. They do not write canonical memory directly.
brigade work import validate imports.jsonl
brigade work import ingest imports.jsonl
brigade work import memory-care
brigade work import chat-sweep
brigade work import triage
brigade work import promote --all --source memory-care --kind task
```

`validate` checks a JSONL file without writing. `ingest` appends valid records into `.brigade/work/imports/inbox.jsonl`, skipping duplicate pending records with the same source, kind, and normalized text. `memory-care` reads `memory/cards/decay/refresh-queue.json` and converts queued cards into task imports.
`validate` checks a JSONL file without writing. `ingest` appends valid records into `.brigade/work/imports/inbox.jsonl`, skipping duplicate pending records with the same source, kind, and normalized text. `memory-care` reads `memory/cards/decay/refresh-queue.json` and converts queued cards into task imports. `chat-sweep` reads `.brigade/chat-memory-sweeps/latest.json` and converts sweep `issues` into incident-oriented imports.

## Record Shape

Expand All @@ -38,6 +39,9 @@ Recommended metadata keys:

- `card_file`: memory card path for memory-care records.
- `reason`: short reason the item was produced.
- `sweep_path`: local chat memory sweep JSON path.
- `issue_title`: title from a chat memory sweep issue.
- `issue_source`: original issue source such as cron, delivery, crawler, or bridge.
- `workspace`: chat workspace or source workspace.
- `channel`: chat channel or surface name.
- `thread`: thread id, message range, or export locator.
Expand Down Expand Up @@ -79,3 +83,46 @@ brigade work import triage
```

The producer writes `task` imports with source `memory-care`, preserving `card_file`, `reason`, and `queue_path` metadata.

## Chat Memory Sweep Producer

The chat-sweep producer reads this summary by default:

```text
.brigade/chat-memory-sweeps/latest.json
```

Minimal sweep shape:

```json
{
"generated_at": "2026-05-26T22:09:00-04:00",
"sessions": {
"listed": 24,
"reviewed": 10,
"durable": 1
},
"issues": [
{
"title": "Cron delivery failure",
"summary": "Recent message delivery failed.",
"kind": "incident",
"source": "cron",
"severity": "warning",
"metadata": {
"surface": "discord",
"local_locator": "crawler://discord/example"
}
}
]
}
```

Run:

```bash
brigade work import chat-sweep
brigade work import triage
```

The producer writes imports with source `chat-memory-sweep`, preserving local locators and summary metadata. It does not import raw transcripts, persisted memory entries, or skipped buckets.
18 changes: 18 additions & 0 deletions src/brigade/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,17 @@ def _build_parser() -> argparse.ArgumentParser:
)
p_work_import_memory_care.add_argument("--dry-run", action="store_true", help="Report without writing imports.")
p_work_import_memory_care.add_argument("--json", action="store_true", help="Print machine-readable JSON.")
p_work_import_chat_sweep = import_sub.add_parser("chat-sweep", help="Import chat memory sweep issues.")
p_work_import_chat_sweep.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to update.")
p_work_import_chat_sweep.add_argument(
"--input",
dest="input_path",
type=Path,
default=None,
help="Chat memory sweep JSON. Defaults to .brigade/chat-memory-sweeps/latest.json under target.",
)
p_work_import_chat_sweep.add_argument("--dry-run", action="store_true", help="Report without writing imports.")
p_work_import_chat_sweep.add_argument("--json", action="store_true", help="Print machine-readable JSON.")
p_work_import_triage = import_sub.add_parser("triage", help="Group pending imports by source and kind.")
p_work_import_triage.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to inspect.")
p_work_import_triage.add_argument("--json", action="store_true", help="Print machine-readable JSON.")
Expand Down Expand Up @@ -684,6 +695,13 @@ def main(argv=None) -> int:
dry_run=args.dry_run,
json_output=args.json,
)
if args.import_command == "chat-sweep":
return work_cmd.import_chat_sweep(
target=args.target,
input_path=args.input_path,
dry_run=args.dry_run,
json_output=args.json,
)
if args.import_command == "triage":
return work_cmd.import_triage(target=args.target, json_output=args.json, limit=args.limit)
if args.import_command == "show":
Expand Down
1 change: 1 addition & 0 deletions src/brigade/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ def build_gitignore_block(selection: Selection) -> str:
".brigade/runs/",
".brigade/scrub-cache/",
".brigade/security/",
".brigade/chat-memory-sweeps/",
".brigade/work/",
GITIGNORE_END,
"",
Expand Down
1 change: 1 addition & 0 deletions src/brigade/templates/depth/workspace.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
{"src": "memory/cards/obsidian-notes.md", "dst": "memory/cards/obsidian-notes.md"},
{"src": "memory/cards/backup-restic.md", "dst": "memory/cards/backup-restic.md"},
{"src": "memory/memory-care.example.json", "dst": ".brigade/memory-care.example.json"},
{"src": "memory/chat-memory-sweep.example.json", "dst": ".brigade/chat-memory-sweep.example.json"},
{"src": "skills/note/SKILL.md", "dst": "skills/note/SKILL.md"},
{"src": "scripts/backup-restic.sh", "dst": "scripts/backup-restic.sh", "mode": "0755"}
],
Expand Down
1 change: 1 addition & 0 deletions src/brigade/templates/harnesses/openclaw.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
{"src": "openclaw/model-aliases.openclaw.json", "dst": ".brigade/openclaw/model-aliases.openclaw.json"},
{"src": "openclaw/ollama-memory-search.openclaw.json", "dst": ".brigade/openclaw/ollama-memory-search.openclaw.json"},
{"src": "openclaw/acp-escalation.openclaw.json", "dst": ".brigade/openclaw/acp-escalation.openclaw.json"},
{"src": "openclaw/memory-sweep-cron.openclaw.json", "dst": ".brigade/openclaw/memory-sweep-cron.openclaw.json"},
{"src": "openclaw/README.md", "dst": ".brigade/openclaw/README.md"}
],
"dirs": [],
Expand Down
10 changes: 10 additions & 0 deletions src/brigade/templates/memory/cards/backup-restic.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,16 @@ Log file path:

Worth wiring into the morning report (`memory/cards/pipeline-standups.md`): grep recent logs for `ERROR` and surface in the briefing. A silent backup that has been failing for three weeks is the second-worst kind of bug.

If chat is part of your operations loop, send backup summaries to the same private status surface you already use for agent reports. Keep the status compact:

- latest NAS snapshot age
- latest cloud snapshot age
- most recent `restic check` result
- prune result
- restore rehearsal date

Route stale or failed backup checks into the Brigade work inbox as incidents. Do not publish real repo names, hostnames, webhook URLs, cloud remote names, or channel ids in public docs.

## Anti-patterns

- **One destination only.** Single point of failure.
Expand Down
17 changes: 17 additions & 0 deletions src/brigade/templates/memory/cards/chat-surface-crawlers.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,23 @@ Each crawler is platform-specific (auth differs, intents differ, rate limits dif
| Platform | Crawler (suggested name) | Status |
|----------|--------------------------|--------|
| Discord | [`discrawl`](https://github.com/solomonneas/discrawl) | Available. Bot-token, SQLite + FTS5, git-snapshot read mode. |
| ClickClack | `clickclack` | Planned adapter. Use its agent-friendly CLI, export command, realtime event API, incoming webhook, and slash command surfaces. |
| Slack | `slackcrawl` | Pattern-name. Bot-token + RTM events; same SQLite shape. |
| WhatsApp | `whatsappcrawl` | Pattern-name. Likely via WhatsApp Business API or Multi-Device session bridge. |
| Telegram | `tgcrawl` | Pattern-name. Bot API + MTProto for history backfill. |
| iMessage | `imescrawl` | Pattern-name. macOS-only; reads from local `chat.db`. |
| Signal | `signalcrawl` | Pattern-name. Signal-cli session export. |
| Matrix | `matrixcrawl` | Pattern-name. Room archive plus event sync. |
| Mattermost | `mattermostcrawl` | Pattern-name. Channel and thread export shape close to Slack. |
| Google Chat / Teams / Feishu | product-specific crawlers | Pattern-name. Enterprise chat adapters should preserve workspace, channel, thread, and user locators. |
| Email (Gmail/IMAP) | `mailcrawl` | Pattern-name. IMAP IDLE for tail, full-folder sync for repair. |

The names above follow the `<platform>crawl` convention used by `discrawl`. Swap them for whatever tool you actually use - the contract matters, not the binary name.

Known channel families worth scanning when present: Discord, Slack, ClickClack, Telegram, WhatsApp, Signal, iMessage, BlueBubbles, Google Chat, Microsoft Teams, Matrix, Mattermost, Nextcloud Talk, Feishu, Line, QQ bot, Zalo, Nostr, IRC, Twitch, Tlon, Google Meet, voice-call transcripts, incoming webhooks, and QA channels.

Do not hardcode this list into business logic. Treat it as a registry that local installs can extend as new OpenClaw, Peter S, Vincent, or other chat plugins appear.

## Discrawl reference (the tested one)

`discrawl` is the canonical implementation:
Expand Down Expand Up @@ -83,6 +91,15 @@ discrawl query --since "$since" --channel "#decisions" --format json |

The scanner writes one Memory Handoff per durable finding. The conservative ingester routes those handoffs into cards / runbooks / learnings the same way it routes anything else.

For operational failures found during the same pass, write a compact sweep file and import the issues into the daily inbox:

```bash
brigade work import chat-sweep --input .brigade/chat-memory-sweeps/latest.json
brigade work import triage
```

This makes delivery failures, stale bridge jobs, and crawler gaps visible without promoting raw chat content.

## Privacy boundary

Chat archives are *intimate*. The crawler's local SQLite often contains:
Expand Down
34 changes: 27 additions & 7 deletions src/brigade/templates/memory/cards/memory-scanner.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ A real implementation (typical nightly cadence):

1. **List recent sessions.** Last 12-24 hours, across all harnesses connected to canonical memory.
2. **Skip noise.** Cron-spawned sessions, heartbeat/reminder-only sessions, empty subagent shells, pure delivery mirror / announce-only sessions.
3. **Prioritize real human-facing sessions first.** Discord, WhatsApp, Telegram, Slack, manual ACP threads. These are where decisions, corrections, and preferences actually land.
3. **Prioritize real human-facing sessions first.** Discord, ClickClack, WhatsApp, Telegram, Slack, manual ACP threads. These are where decisions, corrections, and preferences actually land.
4. **Start with summaries.** Only fetch deeper history for sessions that clearly contain durable decisions, corrections, preferences, project changes, new tooling facts, or new published outputs.
5. **Cap deep review** to the top N most promising sessions (typical: 8) unless there is an obvious reason to exceed it.
6. **Avoid duplication.** Do not re-promote facts that already exist as cards or in daily-log entries.
Expand All @@ -62,29 +62,49 @@ A real implementation (typical nightly cadence):
The scanner reports back on what it did:

```text
Sessions listed: N
Sessions deeply reviewed: N
Sessions with meaningful content: N
Persisted: [bullets]
Skipped: [short bullets]
Net result: [1-3 bullets]
Memory Sweep - YYYY-MM-DD

Sessions: N listed | N reviewed | N with durable content

Persisted:
- card-or-file.md - updated with concise durable change.

Issues:
- Short operational failure or delivery problem that needs review.

Skipped:
- N cron/agent sessions - routine, heartbeat-only, no-reply, or operational noise after signal pass.
- N chat/manual sessions - delivery-only, output artifacts, or no durable preference/tooling/project change.

Net result: Captured durable context and surfaced issues that should not be treated as noise.
```

This gives you an audit trail and a heartbeat for the scanner itself. If "persisted" is empty for a week, either nothing durable happened or the scanner stopped firing.

The scanner should also write a local machine-readable summary at:

```text
.brigade/chat-memory-sweeps/latest.json
```

Use `.brigade/chat-memory-sweep.example.json` as the contract. `brigade work import chat-sweep` imports only the `issues` array into the local work inbox. Raw transcripts, private message bodies, and channel exports stay in crawler archives.

## Scheduling

Common cadence (matches reference cookbook): nightly at quiet hours, after [pipeline-standups](pipeline-standups.md) have run and the day's activity has settled.

Avoid promoting during active sessions because card writes invalidate prefix caches.

Spread recurring jobs around update windows. Do not anchor memory ingest, chat sweep, crawler repair, and updater jobs on the same minute or the same five-minute window. Prefer staggered minute offsets such as `:15` and `:45` for frequent ingest jobs when an updater runs near the top of the hour.

## Implementation surface

`brigade` ships the contract; it does not ship the scanner agent itself. Wire it as:

- a cron job that spawns an isolated agent session with a "review last 12h and persist durable facts" prompt
- the prompt should embed the skip-rules and cost controls above
- output goes either directly to `memory/cards/*.md` (if your harness can write there) or through `.claude/memory-handoffs/` for the conservative ingester to route
- issue summaries also go to `.brigade/chat-memory-sweeps/latest.json` so the daily work loop can review operational failures

## Relationship to Memory Care

Expand Down
Loading
Loading