Skip to content

CONTROL.md

Latest commit

 

History

History
181 lines (150 loc) · 10.5 KB

CONTROL.md

File metadata and controls

181 lines (150 loc) · 10.5 KB
Raw

VibeStudio control plane — vibe CLI, control socket, MCP

Every knob in VibeStudio is scriptable. The GUI is one client; these are the others:

┌─ GUI (webview) ──────────┐
│                          ├─ invoke() ─┐
├─ vibe CLI ───────────────┤            ├─→ Rust core (one AppState) ─→ Python backend / mlx-mtp server
├─ vibe mcp (AI agents) ───┤─ control ──┘
└─ scripts (nc/python) ────┘   socket
  • Control socket — ~/.vibestudio/control.sock (0600), newline-delimited JSON, served by the running app. Auth = file permissions.
  • vibe — bin/vibe (zero-dep python3). Symlink it: ln -s "$PWD/bin/vibe" ~/.local/bin/vibe. When the app is running, commands go through the socket (changes appear live in the GUI). When it's closed, stateless commands fall back to the Python backend directly, and server/model commands manage the mlx-mtp server via ~/.vibestudio/runtime.json.
  • MCP — vibe mcp is a stdio MCP server exposing the same surface as tools. Registered user-scope in ~/.claude.json; or: claude mcp add --scope user vibestudio -- /path/to/vibestudio/bin/vibe mcp

runtime.json — one source of truth for "what is serving"

Whoever starts the server or loads a model (GUI, CLI, MCP, heartbeat) writes ~/.vibestudio/runtime.json {endpoint, api_key, model, server_pid}. On launch the app adopts from it (runtime.json wins over stale UI settings), so a CLI-started server with a CLI-loaded model is exactly what the GUI shows — no orphans, no stub. an optional Bearer key (VIBESTUDIO_API_KEY) is used up automatically when you don't pass --api-key.

Socket protocol

← {"hello":"vibestudio","version":"0.1.0","pid":42257}        on connect
→ {"id":1,"cmd":"load_local_model","args":{"model":"…mxfp4"}}
← {"id":1,"ok":true,"result":{…}}                              (or ok:false + error)

Streaming (start_run/steer_run with "follow":true, or run_follow): event lines {"id":1,"event":{…},"i":N} precede the final result. Multiple requests per connection are fine.

Knob coverage

Knob CLI MCP tool Headless?
status / overview vibe status status ✓ (runtime.json)
launch / quit app vibe app launch|quit app ✓
list models + caps vibe models models ✓
load model (verify) vibe model load [ID|--auto] load_model ✓
server + decoding (standard/mtp/dflash/spec) vibe server start --decoding … --draft-model … server ✓
projects add/rm/rename vibe project(s) … projects ✓
agent run (stream steps) vibe run "…" -p ID run + run_events ✓ (foreground)
steer / cancel / tail vibe steer|cancel|events steer/cancel/run_events app only
composer queue vibe queue add|list|edit|rm|reorder|pop queue ✓
memory files (CLAUDE.md/AGENTS.md) vibe memory list|add|forget|recall|sync|rebuild memory ✓
RAG / knowledge vibe rag index|search|stats|forget rag ✓
settings (sampling/server/composer/tool perms) vibe settings get|set sampling.temperature 0.6 settings ✓
per-model presets vibe settings preset --model M --patch '{…}' settings (preset) ✓
API keys vibe keys new|list|revoke api_keys ✓
app paths (vault/workspace/models dir) vibe config get|set config get only
agent tools / skills store vibe tools / vibe store … capabilities ✓
heartbeat vibe heartbeat status|enable|disable|tick|history heartbeat ✓
triggers (IFTTT, incl. one-shots) vibe trigger list|add|rm|enable|disable triggers ✓
answer the agent's ask_user vibe answer "text" -p ID (auto-prompted in vibe run) answer app only
tool permissions per run vibe run … --perms shell=auto,web=off run (perms) ✓
safety (perm matrix / deny rules / egress / kill-switch) vibe safety pause|resume + vibe settings set safety.… tools.… audit + settings ✓
audit trail vibe audit [-n 25] audit ✓
usage ledger + budgets vibe usage + vibe settings set budgets.… usage + settings ✓
computer use (screen/mouse/keyboard/AppleScript/Chrome) vibe run … --perms computer=auto · vibe settings set tools.computer ask run (perms) + settings ✓
browser automation (Playwright + browser-use, local model) vibe run … --perms browser=auto · vibe settings set tools.browser ask run (perms) + settings ✓

Positionals come before flags: vibe queue add "text" -p proj.

Heartbeat & triggers (the proactive layer)

OpenClaw-style: HEARTBEAT.md (~/.vibestudio/HEARTBEAT.md) is a user-editable checklist; every every_min (within active_hours) an agent run reads it and acts. If nothing needs doing the agent answers HEARTBEAT_OK and the run is marked quiet — proactive, never spammy. A busy project defers the beat to the next tick.

vibe heartbeat enable -p "VS Demo" --every 30 --hours 08:00-23:00
vibe trigger add --kind file --glob "inbox/*.md" -p notes \
     --prompt "Summarize the new inbox note into digest.md"
vibe trigger add --kind daily --at 09:00 -p planner --prompt "Draft my morning brief"
vibe trigger add --kind command --command "git -C ~/work status --porcelain | grep ." \
     --fires-on exit0 -p work --prompt "There are uncommitted changes — list them"

Trigger kinds: once (at_ts / --in-min; pruned after firing) · interval (every_min) · daily (at HH:MM) · file (glob changed; baselines silently on first check) · command (probe; fires on exit-0 edge or output_change). The app evaluates every minute; vibe heartbeat tick is the same evaluation headless (cron/launchd-friendly). Firings are normal agent runs: steps stream to the GUI/CLI and are captured to Obsidian, with origin heartbeat:<id>.

All of it has a GUI home — Settings → Automation: heartbeat knobs (cadence, active hours, project, HEARTBEAT.md editor, Check now), the trigger list with an add form, and recent firings. And the agent can schedule from inside any chat via its schedule tool — "remind me in 20 minutes…", "every morning at 9…", "when the build log changes…" — those land in the same triggers list.

Computer use (drive the Mac)

When the Computer use capability is on (Settings → Privacy & Safety, or --perms computer=auto), the agent gets four macOS-control tools — and the system prompt is seeded with real machine facts (macOS version, arch, shell, screen size) so it uses Mac/BSD idioms instead of guessing:

  • screenshot — screencapture downscaled to logical points; the PNG is fed back to the model as an image_url, so the VLM literally sees the screen. Click coordinates map 1:1 to the image.
  • computer — mouse + keyboard via JXA/CoreGraphics CGEvents and System Events (no cliclick needed): move / click / double / right / scroll / type / key.
  • applescript — the reliable semantic path: Finder file ops, launching apps, driving menus, reading app state.
  • chrome — browser use via Chrome's AppleScript dictionary: navigate, run JS in the active tab (click/fill/scrape), read URL+title+text, list tabs.

Everything is gated by the computer permission (off by default — it controls the whole machine), honors Ask (Allow/Deny in chat), and is written to the audit trail. Zero third-party deps; needs the macOS TCC grants (Screen Recording, Accessibility, Automation) for whatever process runs the agent.

Browser automation (Playwright + browser-use)

Separate browser capability (also off by default) for a real automated Chromium — distinct from the user's own Chrome — running entirely on the local model:

  • browser tool, action: "task" — hand a natural-language web task to an autonomous browser-use agent pointed at the local mlx-mtp endpoint; it navigates, clicks, types and reads on its own, then reports back. One subprocess per task.
  • action: "run" — a deterministic Playwright interaction in one call: open a URL, optionally run [{click},{fill},{press},{wait}] steps, then read the page or a selector's text (+ optional screenshot).

Isolated in its own venv (.venv-browser, see backend/requirements-browser.txt) so Playwright/browser-use never perturb the inference venv; the agent shells out to VIBESTUDIO_BROWSER_PYTHON. Verified live on the local 27B: it opened Hacker News and reported the top story, and the Playwright path extracts page content headless.

vibe run "create vibe-notes.txt on my Desktop, rename it to notes2.txt" -p proj --perms computer=auto
vibe run "open chrome to example.com and tell me the page title" -p proj --perms computer=auto
vibe run "take a screenshot and describe what's on screen" -p proj --perms computer=auto

Generative UI (ask_user)

The agent can pause a run and put a real question to the user: the timeline renders the options as buttons (+ an optional free-text box), and the answer is fed back into the blocked run (answer_run → the sidecar's stdin). vibe run prompts on the terminal when following interactively; MCP agents poll run_events for a step with kind ask and reply via the answer tool. Automation runs never get the tool (nobody is there to answer); unanswered asks time out after ~5 minutes and the agent proceeds with its best judgment.

Memory two-way sync

The markdown files are the database; Zvec is a derived index:

  • writes go through the md first (memory add/forget, the Settings → Memory Edit files… editor, vibe memory edit via $EDITOR), then mirror into the index — managed-section bullets as individual units plus the user's own prose in paragraph chunks, so whole-file edits are recallable too;
  • hand-edits in any editor are picked up by mtime on every sync/recall;
  • files are enumerated for the global scope, workspace checkouts, and curated projects (projects.json) — memory lives at each project's real path;
  • vibe memory show prints the raw file; vibe memory rebuild reconstructs the index purely from the files;
  • recall merges memory entries (kind=memory) with captured run notes (kind=run, incremental append-only index at ~/.vibestudio/recall).

Testing

.venv/bin/python backend/test_vibe_ctl.py     # 35 checks: socket protocol, MCP,
                                                   # heartbeat evaluator, memory sync
.venv/bin/python backend/test_vibestudio.py   # 17 checks: backend e2e