Skip to content

Commit 1dd77ac

Browse files
author
bryanjonas
committed
Apply pending updates and tighten gitleaks YAML policy
1 parent 5c5381a commit 1dd77ac

31 files changed

Lines changed: 1878 additions & 184 deletions

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,8 @@ memories/*.yaml
2525
superpowers/*.yaml
2626
!superpowers/example.yaml
2727
secrets/*.yaml
28-
!secrets/example.yaml
28+
secrets/*.env
29+
!secrets/example.env
2930

3031
# Projects — only example.yaml is committed
3132
projects/*.yaml

.gitleaks.toml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,15 @@ commits = [
1111
"0dafcb4028f8c81cd4fa8bef9cb0f41d9f9430a5",
1212
"02a28b0c9fc2df3973a8951e082b684fbd73b726",
1313
]
14+
15+
[[rules]]
16+
id = "non-example-yaml-file"
17+
description = "Flag YAML files that are not named example.yaml"
18+
path = '''(?i)\.yaml$'''
19+
regex = '''(?s).+'''
20+
21+
[rules.allowlist]
22+
description = "Allowlist example.yaml files"
23+
paths = [
24+
'''(^|/)example\.yaml$''',
25+
]

CLAUDE.md

Lines changed: 94 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Bareclaw — CLAUDE.md
22

3-
Self-hosted AI agent platform. Local Ollama LLM (optional OpenAI) exposed via web UI, Telegram, webhooks, and cron jobs. Single asyncio event loop runs all subsystems concurrently.
3+
Self-hosted AI agent platform. Local Ollama LLM (optional OpenAI) exposed via web UI, Telegram, webhooks, and deterministic scheduled jobs. Single asyncio event loop runs all subsystems concurrently.
44

55
## Running
66

@@ -13,7 +13,28 @@ python main.py
1313

1414
No test suite. Verify changes by running the app and exercising the affected interface.
1515

16-
## Docker
16+
## Deployment
17+
18+
### Systemd (bare-metal)
19+
20+
Runs as a systemd user service — agents have direct access to the host filesystem within their configured workspace.
21+
22+
```bash
23+
mkdir -p ~/.config/systemd/user
24+
cp bareclaw.service ~/.config/systemd/user/
25+
systemctl --user daemon-reload
26+
systemctl --user enable --now bareclaw
27+
journalctl --user -u bareclaw -f
28+
```
29+
30+
To keep running at boot without a login session, a sudoer must run once:
31+
```bash
32+
sudo loginctl enable-linger bareclaw
33+
```
34+
35+
### Docker
36+
37+
Agent file access is limited to volumes explicitly mounted into the container. To expose additional host paths, add volume mounts to `docker-compose.yml` and update the agent's `workspace:`.
1738

1839
```bash
1940
mkdir -p workspace # agent CLI sandbox (host-side)
@@ -44,13 +65,13 @@ main.py
4465
└── db.init_db() # SQLite at data/bareclaw.db
4566
└── build clients from config.providers # OllamaClient or OpenAIClient per provider
4667
└── build_app(config, clients) # FastAPI + WebSocket chat + dynamic webhook routes
47-
└── scheduler (APScheduler) # cron jobs → agent runs → optional Telegram notify
68+
└── scheduler (APScheduler) # cron jobs → project tasks or commands → optional Telegram notify
4869
└── telegram bot (optional)
4970
```
5071
5172
**Agentic loop** (`bareclaw/core/agent.py`): system prompt + messages → LLM → if tool_calls → dispatch → loop; capped by `max_iterations`.
5273
53-
**Multi-provider LLM** (`bareclaw/core/llm.py`): `OllamaClient` and `OpenAIClient` both normalise to the same canonical message dict. `config.providers` is a named map; `main.py` builds one client per provider at startup. `agent.py` is provider-agnostic — it looks up `clients[agent.provider]` by id. Any number of Ollama instances, OpenAI-compatible servers (LM Studio, vLLM, llama.cpp), or the real OpenAI API can be configured simultaneously.
74+
**Multi-provider LLM** (`bareclaw/core/llm.py`): `OllamaClient` and `OpenAIClient` both normalise to the same canonical message dict. `config.providers` is a named map; `main.py` builds one client per provider at startup. `agent.py` is provider-agnostic — it looks up `clients[agent.provider]` by id. Any number of Ollama instances, OpenAI-compatible servers (LM Studio, vLLM, llama.cpp, OpenRouter.ai), or the real OpenAI API can be configured simultaneously.
5475
5576
## Key Files
5677
@@ -61,13 +82,17 @@ main.py
6182
| `bareclaw/config.py` | Dataclass config loader |
6283
| `bareclaw/db.py` | SQLite schema + log helpers |
6384
| `bareclaw/core/llm.py` | OllamaClient + OpenAIClient |
64-
| `bareclaw/core/agent.py` | Agentic loop (`run_agent`, `run_agent_stream`) |
85+
| `bareclaw/core/agent.py` | Agentic loop (`run_agent`, `run_agent_stream`), system prompt building with auto-injection |
86+
| `bareclaw/core/task_runner.py` | Project task execution with auto-injection of project memories |
6587
| `bareclaw/core/tools.py` | Tool registry + schemas |
88+
| `bareclaw/core/memory.py` | Memory loading, keyword matching, read/write tools |
89+
| `bareclaw/core/superpowers.py` | Superpower loading, keyword matching, bootstrap interpolation |
90+
| `bareclaw/core/projects.py` | Project loading, keyword matching, task resolution, bootstrap interpolation |
6691
| `bareclaw/executor/cli.py` | `run_command` / `read_file` (workspace-sandboxed) |
6792
| `bareclaw/scheduler/jobs.py` | APScheduler cron dispatch |
6893
| `bareclaw/webhooks/handler.py` | Dynamic webhook route registration |
6994
| `bareclaw/telegram/bot.py` | Telegram bot + per-chat agent sessions |
70-
| `bareclaw/web/routes.py` | FastAPI routes + WebSocket chat |
95+
| `bareclaw/web/routes.py` | FastAPI routes + WebSocket chat + bootstrap endpoints |
7196
| `bareclaw/web/auth.py` | API key middleware (Bearer / cookie / WS query param) |
7297
7398
## Config-Driven Entities
@@ -94,12 +119,27 @@ max_iterations: 10
94119
```yaml
95120
id: my-cron
96121
schedule: "0 * * * *" # 5-field cron expression
97-
agent: my-agent
98-
command: "df -h" # optional; output prepended to prompt
99-
prompt: "Analyse the above and alert if ..."
122+
project: my-project
123+
task: check-system
100124
notify_telegram: false
101125
```
102126
127+
Exactly one target must be defined per cron job:
128+
- `project` + `task` for a scheduled project task
129+
- `command` for an explicit shell command, optionally with `workspace` and `timeout`
130+
131+
Command example:
132+
```yaml
133+
id: disk-check
134+
schedule: "0 * * * *"
135+
command: "df -h"
136+
workspace: ~/workspace
137+
timeout: 30
138+
notify_telegram: true
139+
```
140+
141+
Project-task crons resolve the referenced task directly in Python and run that task prompt using `task.agent → project.agent → config.default_agent`. Command crons execute the configured shell command directly. Cron jobs do not self-call the HTTP API.
142+
103143
### Webhook (`webhooks_config/<id>.yaml`)
104144
```yaml
105145
id: my-webhook
@@ -139,7 +179,7 @@ content: |
139179

140180
Core module: `bareclaw/core/memory.py` — `load_all()`, `load_one()`, `find_relevant()`, `save()`
141181

142-
## Superpowers (`superpowers/<id>.yaml` + `secrets/<id>.yaml`)
182+
## Superpowers (`superpowers/<id>.yaml` + `secrets/<id>.env`)
143183

144184
Named external service capabilities bundling config, secrets, and an optional bootstrap prompt. Loaded fresh on each agent call.
145185

@@ -160,27 +200,33 @@ bootstrap_prompt: |
160200
bootstrap_agent: default # optional; defaults to app's default_agent
161201
```
162202

163-
**`secrets/<id>.yaml`** (always gitignored — flat key/value):
164-
```yaml
165-
token: "your-token-here"
203+
**`secrets/<id>.env`** (always gitignored — KEY=VALUE format):
204+
```dotenv
205+
token=your-token-here
166206
```
167-
The filename must match the superpower `id`. Consider `chmod 600 secrets/<id>.yaml`.
207+
The filename must match the superpower `id`. Consider `chmod 600 secrets/<id>.env`.
168208

169-
**Auto-injection**: `_build_system_content()` in `bareclaw/core/agent.py` keyword-matches user messages against all superpowers and appends matching ones (including secrets) to the system prompt under `## Available superpowers`.
209+
**Auto-injection**: `_build_system_content()` in `bareclaw/core/agent.py` keyword-matches user messages against all superpowers and appends matching ones to the system prompt under `## Available superpowers`. Config values are shown; secrets are represented as the file path + variable names only (values never enter LLM context). Example injection:
210+
```
211+
Credentials: source /path/to/secrets/homeassistant.env # exports: token
212+
```
213+
The agent uses `run_command` to source the file: `source /path/to/secrets/homeassistant.env && curl -H "Authorization: Bearer $token" ...`
214+
215+
**Bootstrap**: clicking "Bootstrap Memory" in the `/superpowers` UI POSTs to `/api/superpowers/{id}/bootstrap`. The server interpolates `{key}` placeholders in `bootstrap_prompt` with merged config+secrets values, then runs the bootstrap agent. The agent typically uses `run_command` (curl) + `write_memory` to document findings.
170216
171-
**Bootstrap**: clicking Bootstrap in the `/superpowers` UI POSTs to `/api/superpowers/{id}/bootstrap`. The server interpolates `{key}` placeholders in `bootstrap_prompt` with merged config+secrets values, then runs the bootstrap agent. The agent typically uses `run_command` (curl) + `write_memory` to document findings.
217+
**Provider API keys** also use `.env` format — `secrets/<provider-id>.env` with `api_key=sk-...`. Loaded by Python at startup; the LLM never sees them.
172218
173219
**Tools** (always available to all agents — no YAML config needed):
174220
- `list_superpowers` — returns id, name, description, keywords for all superpowers
175-
- `read_superpower(id)` — returns full config + secrets so the agent can use them
221+
- `read_superpower(id)` — returns config values + credentials file path and variable names
176222
177-
`superpowers/example.yaml` and `secrets/example.yaml` are committed to git; all other files in both dirs are gitignored. Both dirs are mounted as volumes in Docker.
223+
`superpowers/example.yaml` and `secrets/example.env` are committed to git; all other files in both dirs are gitignored. Both dirs are mounted as volumes in Docker.
178224
179225
Core module: `bareclaw/core/superpowers.py` — `load_all()`, `load_one()`, `find_relevant()`, `_load_secrets()`, `interpolate()`
180226
181227
## Projects (`projects/<id>.yaml`)
182228
183-
Multi-component workflows the agent has explored and can execute. Each project defines named **tasks** — runnable prompts triggerable from the `/projects` UI or by agents via tools. Loaded fresh on each agent call (no restart needed). Safe to commit (no secrets).
229+
Multi-component workflows the agent has explored and can execute. Each project defines named **tasks** — runnable prompts triggerable from the `/projects` UI, by cron schedules, or by agents via tools. Loaded fresh on each agent call (no restart needed). Safe to commit (no secrets).
184230
185231
```yaml
186232
id: home-network-security
@@ -191,9 +237,9 @@ keywords:
191237
- pcap
192238
- network security
193239
agent: default # default agent for tasks; falls back to config.default_agent
194-
memories: # related memory IDs shown in UI and injected into system prompt
195-
- home-network-architecture
196-
- pcap-pipeline-process
240+
memories: # auto-injected into task context when tasks execute
241+
- home-network-security-runbook
242+
- home-network-troubleshooting
197243
tasks:
198244
- id: run-pipeline
199245
name: "Run Pipeline"
@@ -206,19 +252,42 @@ tasks:
206252
prompt: |
207253
Check the security dashboard for anomalies in the last 24 hours.
208254
agent: "" # optional per-task agent override
255+
bootstrap_prompt: |
256+
You are bootstrapping the project "{name}" (ID: {id}).
257+
258+
Description: {description}
259+
260+
Available tasks: {tasks}
261+
262+
Your goal is to RUN these tasks and document practical operational knowledge.
263+
Create a memory called '{id}-runbook' using write_memory with execution flow,
264+
file locations, dependencies, timing, and troubleshooting tips.
265+
bootstrap_agent: "" # optional; defaults to project.agent or config.default_agent
209266
```
210267

211-
**Auto-injection**: `_build_system_content()` keyword-matches user messages against all projects and appends matching ones under `## Relevant projects`, including task summaries and referenced memory IDs.
268+
**Auto-injection (chat context)**: `_build_system_content()` keyword-matches user messages against all projects and appends matching ones under `## Relevant projects`, including task summaries and referenced memory IDs.
269+
270+
**Auto-injection (task execution)**: When a task runs via `run_project_task()` in `bareclaw/core/task_runner.py`, all memories listed in the project's `memories:` field are automatically loaded and injected into the task's user prompt under `## Project Knowledge`. This means:
271+
- Tasks always have access to the project's operational knowledge (runbooks, troubleshooting guides)
272+
- No need for agents to explicitly call `read_memory()`
273+
- Cron jobs get the same context as manual runs
274+
275+
**Task execution**: clicking Run in the `/projects` UI POSTs to `/api/projects/{id}/tasks/{task_id}/run`. Cron jobs also resolve tasks by `project` + `task` and run the same prompt path. Agent resolved as `task.agent → project.agent → config.default_agent`.
276+
277+
**Bootstrap**: clicking "Bootstrap Runbook" in the `/projects` UI POSTs to `/api/projects/{id}/bootstrap`. The server interpolates `{key}` placeholders in `bootstrap_prompt` (available: `{id}`, `{name}`, `{description}`, `{agent}`, `{memories}`, `{tasks}`) using `proj_mod.interpolate()`, then runs the bootstrap agent. The agent typically executes tasks using available tools and uses `write_memory` to create a `{id}-runbook` memory.
212278

213-
**Task execution**: clicking Run in the `/projects` UI POSTs to `/api/projects/{id}/tasks/{task_id}/run`. Agent resolved as `task.agent → project.agent → config.default_agent`.
279+
The Bootstrap Runbook button:
280+
- Only appears if `bootstrap_prompt` is defined
281+
- Hides automatically once `memories/{id}-runbook.yaml` exists (checked via `proj_mod.has_runbook()`)
282+
- Reappears if the runbook memory is deleted
214283

215284
**Tools** (always available to all agents — no YAML config needed):
216285
- `list_projects` — returns id, name, description for all projects
217286
- `read_project(id)` — returns full project details including tasks and prompts
218287

219288
`projects/example.yaml` is the only project file committed to git; all others are gitignored.
220289

221-
Core module: `bareclaw/core/projects.py` — `load_all()`, `load_one()`, `find_relevant()`
290+
Core module: `bareclaw/core/projects.py``load_all()`, `load_one()`, `load_task()`, `find_relevant()`, `interpolate()`, `has_runbook()`
222291

223292
## Adding a New Tool
224293

0 commit comments

Comments
 (0)