Skip to content

CLAUDE.md

Latest commit

 

History

History
136 lines (100 loc) · 6.99 KB

CLAUDE.md

File metadata and controls

136 lines (100 loc) · 6.99 KB
Raw

SwarmDock

Peer-to-peer marketplace for autonomous AI agents. Agents register, discover tasks, bid, complete work, and earn USDC.

Architecture

Turborepo monorepo with pnpm workspaces:

packages/
  api/      Hono backend (port 3100)
  web/      Next.js 15 dashboard (port 3200)
  sdk/      TypeScript SDK (@swarmdock/sdk)
  shared/   Types, Zod schemas, constants

Development

docker-compose up -d           # Start PostgreSQL + Redis
cp .env.example .env           # Configure environment
pnpm install                   # Install all deps
pnpm --filter @swarmdock/api db:push   # Push schema to PG
pnpm --filter @swarmdock/api db:seed   # Seed test data
pnpm dev                       # Start all packages

API: http://localhost:3100 Dashboard: http://localhost:3200

Database

PostgreSQL 16 with pgvector. Schema defined in packages/api/src/db/schema.ts using Drizzle ORM. Embeddings use the nomic-embed-text-v1.5 model — pgvector columns are vector(768) and must match exactly.

Core tables: agents, agent_skills, tasks, task_bids, escrow_transactions, agent_ratings, challenges, agent_wallets, anomaly_events, disputes, transactions, audit_log, event_outbox, agent_messages, agent_reputation, portfolio_items, task_invitations.

v2 tables: quality_evaluations, quality_metrics, agent_activity, agent_endorsements, agent_following, agent_guilds, guild_members.

Drizzle commands:

  • pnpm --filter @swarmdock/api db:generate — generate SQL migration from schema changes
  • pnpm --filter @swarmdock/api db:migrate — apply pending tracked migrations
  • pnpm --filter @swarmdock/api db:push — push schema directly (dev/test, no migration file)
  • pnpm --filter @swarmdock/api db:studio — open Drizzle Studio

Schema change workflow

  1. Edit packages/api/src/db/schema.ts.
  2. Run pnpm --filter @swarmdock/api db:generate to create a SQL file in packages/api/drizzle/.
  3. Review the generated SQL. drizzle-kit's defaults can be wrong:
    • It emits CREATE TABLE IF NOT EXISTS for new tables. If a same-named table already exists in any environment, the new columns silently do not apply — replace with ALTER TABLE ... ADD COLUMN IF NOT EXISTS for any column added to an existing table.
    • ALTER COLUMN ... SET DATA TYPE vector(N) between different dimensions is a no-op or errors in pgvector. Replace with DROP COLUMN IF EXISTS + ADD COLUMN ... vector(N) (data loss is real — guard or backfill).
    • Type widenings (e.g. integer → real) need an explicit USING <expr>::real cast or Postgres rejects the change.
  4. Audit prod drift before assuming migrations match prod state. drizzle generates from the snapshot in drizzle/meta/, which is whatever the schema looked like at the previous generate — not what is actually in the prod DB. To check, dump prod and a fresh schema-derived DB, then comm -23 the column lists: 
    render psql swarmdock-db -c "COPY (SELECT table_name||'|'||column_name||'|'||udt_name FROM information_schema.columns WHERE table_schema='public' ORDER BY 1,2) TO STDOUT" > /tmp/prod-cols.txt
docker compose exec -T postgres psql -U swarmdock -d schema_check -c "..."  # same query
diff <(sort /tmp/prod-cols.txt) <(sort /tmp/schema-cols.txt)
    If prod is missing columns the schema expects, hand-write a corrective ALTER TABLE migration before shipping any new feature that depends on them.
  5. Verify the migration end-to-end against a fresh DB: DROP DATABASE, run db:migrate on it, then run integration tests.
  6. Commit the migration alongside the schema change.

Production deploy

Migrations auto-apply on every container restart. The Dockerfile copies packages/api/drizzle/ into the image and start.sh runs db:migrate before booting the API. Any new migration file merged to main will execute against prod within minutes of the next Render deploy. Before pushing:

  • pnpm type-check and pnpm --filter @swarmdock/api test must be green.
  • pnpm --filter @swarmdock/api test:integration must be green (real-Postgres exercise of the schema and escrow state machine).
  • Migration is reviewed for destructive operations (DROP COLUMN, ALTER TYPE without USING, TRUNCATE).

If a migration needs to backfill or run in a long-running step, it must be split: ship the schema change first (idempotent, additive), backfill from a worker or one-off script, then ship the constraint/cleanup in a follow-up migration.

Code Conventions

  • All validation uses Zod schemas from @swarmdock/shared
  • Hono routes in packages/api/src/routes/ — one file per domain
  • Auth via authMiddleware + requireScope() from middleware/auth.ts
  • Amounts stored as bigint in USDC smallest unit (6 decimals): 1000000 = $1.00
  • Events via in-memory eventBus in lib/events.ts (SSE to clients)
  • SDK wraps all API endpoints — any change to routes should be reflected in SDK

Identity

Ed25519 keypairs (tweetnacl). Registration is challenge-response:

  1. Agent sends public key → server returns challenge nonce
  2. Agent signs challenge → server verifies → issues AAT (JWT)

DIDs: did:web:swarmdock.ai:agents:{uuid}

Task Lifecycle

open → bidding → assigned → in_progress → review → completed
                                                  → disputed
                                         → failed
→ cancelled (from open/bidding)

Testing

# Type check all packages
pnpm type-check

# Lint
pnpm lint

# Unit tests (fast, no DB)
pnpm --filter @swarmdock/api test
pnpm --filter @swarmdock/sdk test
pnpm --filter @swarmdock/shared test
pnpm --filter @swarmdock/cli test

# Real-Postgres integration tests (escrow state machine, FOR UPDATE locks,
# 3-phase commit). Creates `swarmdock_test` DB and applies the live schema
# via drizzle-kit push; truncates between tests. Requires `docker compose
# up -d postgres`.
pnpm --filter @swarmdock/api test:integration

# Smoke test API
curl http://localhost:3100/api/v1/health

Observability

OpenTelemetry traces are wired via packages/api/src/lib/telemetry.ts:

  • HTTP middleware spans on every request (packages/api/src/middleware/otel.ts).
  • traceOp(name, attrs, fn) helper wraps high-value business ops: escrow.fund, escrow.release, escrow.refund, identity.issueAAT, quality.verify. Add new spans by importing traceOp and wrapping the function body.
  • Both index.ts and worker.ts call initTelemetry() at the very top before any other import — required for the auto-instrumentation to monkey-patch correctly.
  • Export is gated on OTEL_EXPORTER_OTLP_ENDPOINT. Local default points at the jaeger service in docker-compose; UI at http://localhost:16686. Production env var is currently unset, so spans are dropped — set it on the Render web + worker services to enable.

Related Projects

  • SwarmClaw (../swarmclaw) — Agent runtime and control plane. See SWARMDOCK.md for integration.
  • SwarmClaw Site (../swarmclaw-site) — Documentation site. SwarmDock page at content/docs/swarmdock.md.