- Bun workspace under
packages/*; TS source is insrc, tests in__tests__. - Main packages:
coreshared types/utils,plugin-apinative plugin contracts,plugin-hostdeterministic plugin composition,servergateway + embedded runtime,agent-workerLobu agent execution,connectorsbuilt-in connectors,owlettofrontend submodule. - Before editing a package, read its nearest
AGENTS.mdif present. docs/GOTCHAS.mdcollects the mechanical traps (build, SQL, testing, submodule, browser) that have each cost a debugging session. Skim the section for the surface you are touching before you start, and grep it when something looks inexplicable.
- Never write to
~/Code/lobudirectly — work in a worktree. Runmake task-setup NAME=<slug>first, and nevergit switchin the main checkout. - One branch = one concern (not one file). Tangential task → commit/push/open current work, then branch fresh from
main. Split only for genuinely independent concerns or an unreviewable diff. - Multi-replica correctness is mandatory. Never put shared required state in an in-memory Map/singleton another replica must read or mutate. If a feature relies on cross-pod visibility, use Postgres-mediated state/signal.
eventsis append-only. NeverDELETE FROM events; tombstone/supersede instead.events.idis a stored-version id, not stable source identity. Connector resyncs can supersede a row and allocate a new id for the same source item. Cross-sync identity and dedupe must use the connector's stableorigin_id(scoped to its connection/source) or an explicit domain key, neverevents.id.- Request paths never aggregate history. No
GROUP BY,DISTINCT ON, per-rowregexp_*, or leading-wildcardLIKEover an append-only / run-history table (events,agent_transcript_snapshot,session_calls, …) on a user-facing read path. History only grows, so read-time aggregation cost grows with it while the answer stays bounded. Materialize the row or counter at WRITE time and read it back by index; run the one-time aggregation in a migration, never per request. Aggregating bounded config tables (connections,watchers,agent_users) is fine. - Never bulk-delete prod
organizationrows, including zero-activity ones — empty-looking orgs are frequently real human signups. Surface them one at a time for confirmation. - Workers never receive real credentials. They may receive only placeholders/proxied access. The only exceptions are device-pinned connectors and short-lived provider-derived credential leases (see
packages/server/AGENTS.md); a durable stored credential is never one of them. - Default to static
import. New dynamic imports require measured cost justification here or in the package AGENTS plus a rationale comment at the call site. Tests may dynamically import after mocks.- Node-version gate exceptions:
packages/cli/bin/lobu.jsdefers the existing CLI graph without duplicating it, andpackages/server/src/server-entry.tsadds an 842-byte gate in front of the 4.34 MB server graph (measured 2026-07-24). Both call sites must stay dependency-free until their checks pass.
- Node-version gate exceptions:
- Bug fixes require red→fix→green evidence, with both outputs pasted into the PR body. If you cannot reproduce, stop and report the dead end — do not ship a speculative fix.
- Mutation checks require proof that the mutation landed: assert the old pattern exists, verify a countable source change, and confirm the focused test fails before restoring the fix and rerunning green. A green test after a no-op mutation proves nothing.
- Never report done off a green typecheck. Boot it, exercise every branch you touched, clean up test data. If "compiles" is all you ran, say so explicitly.
- Gates are non-negotiable (sequence in "Ship a change" below).
make pre-prbuilds first — that build step is what protects against a stale-dist false green.make reviewis an LLM verdict only: it does NOT run typecheck/knip/tests and is not proof CI will pass. - The agent-facing product name is Behavior;
watcheris internal engine/DB vocabulary. It must not appear in MCP tool schemas/names/descriptions, ClientSDK discovery metadata, or the connector-sdk public reaction contract —make pre-prfails on it. Internal identifiers (actingWatcherId, table/column names, comments) are fine. make review-fixis the unposted fixer pass and runs BEFORE the firstmake review; verify its diff and commit it. Never iteratemake reviewas a find-fix loop — each posted round costs a review + CI cycle.- A UI change owes visual proof. For an Owletto pointer PR,
make ui-reviewpublishes the before/after comparison on the exact merged Owletto PR and links it from the parent. Publishing the proof is what the gate asserts — review it like any other change on the PR. A changed pointer republishes it. - Fix the class, not the instance: on the first hit, grep every other instance and fix in one pass — a diff-scoped reviewer only ever finds the next one. Same root cause twice = stop reviewing, grep-enumerate. A class-wide fix is in scope for the branch that found it and does not count as scope creep.
- Fail closed on durable dispatch/delivery state. When a durable coordination or delivery operation fails, its caller must not reinterpret the error as proceed/deliver/skip. Propagate a retry, defer, or terminal-failure outcome with a visible log; if the operation may already have succeeded, reconcile idempotently before retrying. Treat lock timeouts, pool errors, and ambiguous or expired rows as failures, not as "nothing changed."
- Coordination design brake. A change that needs a second lock, a second retry/deferral budget, or a prose termination argument to explain why it halts is patching a misplaced check — stop and re-derive where the decision belongs before adding mechanism. Find the chokepoint that already serializes the action (e.g. worker dispatch is serialized by
job-routeron the worker-SSE-owner pod) and put the check there; prefer deleting the state transition over coordinating it, and reuse queue-native retry over hand-rolled hold/requeue. - Stage by explicit path; never commit the whole tree of a worktree that hosted other work. A commit is a snapshot — stale file copies silently revert already-merged PRs. After commit/rebase,
git diff --name-only origin/main...HEADmust equal the task's intended file list; extra files = stop. Verify a "reverts merged work" review finding by diff direction againstorigin/main, never dismiss it as merge-base noise.
make task-setup NAME=<slug>→ work in.claude/worktrees/<slug>/.- Reproduce the bug (red) before fixing it. Fix, then prove green.
- Test:
make test-unit(no DB) /make test-integration(needsDATABASE_URLwith pgvector). One file:bun test <path>, orcd packages/server && bunx vitest run <path>for vitest suites. make pre-pr— build, typecheck, knip, lint, exposed-surface naming.make review-fixon the settled diff and verify what it changed — it edits the working tree, so re-read the files before trusting them.- Stage by explicit path (
git add -- <paths>, never-A), commit, then confirmgit diff --name-only origin/main...HEADis exactly your intended file list. Extra files = stop. git push -u origin <branch>→gh pr create(fill.github/pull_request_template.md; conventional-commit title, e.g.fix(server): …).make reviewonce on the settled HEAD. It posts thepi-reviewstatus, which is a REQUIRED check. Small safe-class diffs (docs/renames/generated/additive-tests, <100 lines) skip the cross-harness reviewer by default; runREVIEWER_MODE=full make reviewto force the independent review, andREVIEWER_SHADOW=1to measure the skip rule instead of trusting it.make ui-review. It passes non-Owletto changes as not applicable. For an Owletto pointer change, runARTIFACT=<hosted-html-url> make ui-review; it publishes the before/after proof on the exact merged Owletto PR, links it from the parent, and passes. NoARTIFACTmeans no proof and no pass.- Merge squash once CI is green:
gh pr merge <n> --squash --admin. Never--adminpast a check that has not reported. - Prod-visible surface? Verify it live after rollout. Set a self-rescheduling
ScheduleWakeup(~1500s) carrying the rollout gate: poll the deployed SHA, thengit merge-base --is-ancestor "$MERGE_SHA" "$DEPLOYED_SHA"withMERGE_SHA=$(gh pr view <n> --json mergeCommit --jq .mergeCommit.oid). Two ways this gate silently never opens — gate on the squash commit, not your branch head (a squash merge writes a new commit with no ancestry link to the branch, so the branch head exits 1 forever: PR #2280 has identical trees and still fails), and keep that argument order (merge commit first; reversed, it accepts a stale deploy). Run the live check when it lands, and write the result back to memory.
- Do only what was asked. Delete ephemeral files you create. Do not create
*.mdunless asked. - Never introduce a new table or change the database design, API surface, or SDK surface without explicitly surfacing the proposal and receiving user confirmation first.
- Prefer
bun; do not use npm/yarn/pnpm for repo work. - Fix unused params by deleting them, not
_-prefixing. - Never
git stash; use WIP commits and squash later. - Never resolve merge conflicts in the GitHub web UI (it has silently dropped 1000+ lines of feature work). Rebase locally against
origin/main, then diff against the last real feature commit before squashing — except on submodule-pointer branches, where you mustgit merge(seedocs/GOTCHAS.md, "Submodule & cross-repo"). - Run mandated gates automatically; never ask permission to run one.
- Show the diff, not a summary. Before claiming done — on your own work or a dispatched agent's — surface
git statusplusgit diff --stat <base>...HEAD. An agent's self-report is a claim; the diff is the evidence. - UI PRs owe before/after screenshots, and
ghcannot upload images inline. Capture before shots from the unmodified branch and after shots from the same booted app (auth recipe indocs/BROWSER_TESTING.md), base64-embed the PNGs into one self-contained HTML comparison page, publish it as a claude.ai artifact, and pass that URL asARTIFACTtomake ui-review. The gate records it on the Owletto PR and binds human approval to the exact parent-base and Owletto-pointer pair. - Any subagent with write access (Edit/Write/Bash) runs in its own worktree; only grep/read agents share the parent. Two writing agents must NEVER share a worktree — they overwrite each other's uncommitted work.
- Dispatch code-writing agents through
make task-setup NAME=<slug>and brief them with the absolute.claude/worktrees/<slug>/path. Only task-setup does the submodule branch pinning, post-initbun install,.envcopy, and port allocation. - Do repetitive multi-file edits (renames, signature changes, import rewrites) inline or with one script. Reserve subagents for read-only breadth or genuinely isolated parallel work.
- Before adding an env var, grep for the one the codebase already reads. Do not rename existing vars to a "cleaner" convention unasked.
- Deleting code needs structural evidence (dangling import, completed migration, superseded impl). Low prod usage is not a delete signal; trace the workflow, not just the import graph — docs and help text are load-bearing too.
- Absence is never proven by a grep on a working tree. To prove code does not exist, run
git fetch -q origin && git grep <pattern> origin/main— the fetch is load-bearing,origin/mainitself goes stale. Applies to every existence check, including the deletion-evidence rule above. - Slack link pasted (
slack.com/archives/…?thread_ts=) → runscripts/slack-thread-viewer.js "<link>"first. - To drive the user's real logged-in browser, use the paired Owletto extension — recipe in
docs/BROWSER_TESTING.md. Do not assume CDP/browser-auth is required. - Unsure in planning → ask before making conflicting or irreversible choices. Mid-execution, block on a question only for irreversible/destructive actions or decisions that are genuinely the user's; for reversible choices with a clear recommended option, take it and flag the choice in your summary.
- Never poll in the foreground (
sleep/until/whilewait loops, repeatedtail). Run long waits (dev-server boot, CI, deploys) in the background and act on the completion notification. - Prefer DOM reads (
get_page_text,read_page,javascript_tool) over screenshots; screenshot only when visual layout itself is under test. Screenshots are the #1 context-bloat source. - Read a file before editing it, and re-read it after any external change (a fixer pass, another agent, a rebase). Blind edits fail and cost a retry round-trip.
- Read PR status compactly:
gh pr view <n> --json number,title,isDraft,mergeStateStatus,reviewDecisionplusgh pr checks <n> --required. Never--json statusCheckRollup— measured on PR 2280 it is 7520 bytes against 861, 8.7x larger and less legible. - Batch narrow fixes into one commit, verified once at the end of the batch — not per fix. It never justifies skipping a reproducer or a correctness-critical test. For a one-line or config-only change, take one CI snapshot and move on.