← Back to Surla README · Agent Quickstart (AGENTS_QUICKSTART.md) · Acceptance Suite (../orders-eval)
examples/orders is a complete, Golden Path Goa-AI platform that demonstrates why teams build on Surla:
- 0-token transport & schema compilation (
design/*.go→gen/): One compact Goa + Goa-AI design compiles the public HTTP service (orders), the externalcommerceandpaymentsHTTP/JSON service contracts, and the durablerefundagent (orders.refund)—without paying an LLM to write boilerplate handlers, OpenAPI specs, or JSON validators. - Contract-backed security & cost guardrails:
- Anti-spoofing (
Inject+ server lookup scoping):Inject("tenant_id", "customer_id")omitstenant_idandcustomer_idfrom the LLM-visible JSON Schema on every tool inorders.orders(list_orders,continue_orders,issue_store_credit,request_refund) and marks them as server-supplied fields, while theordersservice (internal/orders/service.go) binds the authenticated caller's tenant/customer identity and scopescommerce.get_orderlookups so another customer'sorder_idis rejected. - Cursor pagination (
BoundedResult+ page cap + token budget):BoundedResultdeclares thereturned/truncated/next_cursorcontinuation contract onlist_ordersandcontinue_orders,internal/orders/service.goenforces the page limit (ordersPageSize = 10), and the evaluator'swithin_token_budgetgate catches oversized tool responses. - Human-in-the-loop pause/resume (
Confirmation):request_refunddeclares a design-timeConfirmationgate that pauses the run until a manager holdingrefunds:approvescope approves or denies viaPOST /runs/{run_id}/approval(while customer tokens holding onlyorders:chatreceive403 Forbidden). - Bounded execution (
RunPolicy): Capsorders.refundatMaxToolCalls(8),MaxRecoveryTurns(2), andTimeBudget("2m").
- Anti-spoofing (
- Verification & holdout redaction (
surla verify+examples/orders-eval): The evaluator builds and runsordersandorders-workeras ordinary processes, routes LLM calls through an evaluator-owned gRPC model gateway, kills the worker mid-refund (kill -9) to prove payment idempotency, restarts the API server while a refund awaits manager approval, and enforces a hard token/cost budget that catches seeded payload bloat (TestSeededPayloadBloatFailsTheBudget). A hidden scenario, taggedholdout, is one whose name, checks and logs the builder (the coding agent that writes the change) never sees.
flowchart LR
Customer["Customer / Manager\n(JWT Bearer Token)"] -->|"HTTPS, JSON"| API["Orders API (cmd/orders)\nPublic HTTP API + orders.refund Agent"]
API -->|"SQLite"| RunStore["Run store\nRuns, Transcripts & Pending Approvals"]
API -->|"HTTPS, JSON"| Commerce["Commerce system\nCustomer Order Records"]
API -->|"HTTPS or gRPC Model Gateway"| Model["Language model\nTool Planning & Token Accounting"]
API -->|"Temporal, gRPC"| Workflow["Workflow engine\nAgent Turns & Refund Workflows"]
Worker["Refund worker (cmd/orders-worker)\nIdempotent Refund Execution"] -->|"Temporal, gRPC"| Workflow
Worker -->|"HTTPS, JSON (Idempotency-Key)"| Payments["Payment provider\nStore Credit & Card Refunds"]
| Path | Ownership & Purpose |
|---|---|
design/orders.go |
Public orders HTTP service (POST /chat, GET /runs/{run_id}, POST /runs/{run_id}/approval), the prompts service a live target (section 3) serves to Surla (PUT /prompts/{prompt_id}), orders.refund agent, orders.orders toolset (Inject, BoundedResult, Confirmation, RunPolicy). |
design/deps.go |
External commerce and payments HTTP/JSON service contracts (GET /tenants/{tenant_id}/customers/{customer_id}/orders, POST /tenants/{tenant_id}/refunds). |
design/model.go |
goa.design/model C4 architecture containers and relationships rendered to gen/structurizr/model.json. |
AGENTS_QUICKSTART.md |
Auto-generated contextual map read by coding agents before touching implementation files. |
internal/orders/ |
Authenticated HTTP service, server-side tenant_id/customer_id tool methods, ordersPageSize = 10 pagination, manager approval gate (POST /runs/{run_id}/approval), and the live target's prompt operation (PromptService, PUT /prompts/{prompt_id} requiring the live key Surla passes in SURLA_LIVE_TARGET_KEY). |
internal/assistant/ & internal/refundflow/ |
Refund agent planner, runtime prompt override validation and store updates (assistant.Prompts), and Temporal refund workflow execution. |
internal/runstore/ |
SQLite-backed durable run, transcript, and pending-approval store owned by the Orders API. |
cmd/orders/ & cmd/orders-worker/ |
Public Orders API server and Refund worker binaries verified by surla verify. |
Run the platform's unit tests, Surla's 0-token local architecture preview (surla preview), and Surla's structural Goa/Goa-AI design diff (surla diff) from the repository root:
# Run the orders platform unit tests
(cd examples/orders && go test -race ./...)
# Inspect the compiled C4 topology, agent policies, and tool guardrails at 0 LLM tokens
go run ./cmd/surla preview -dir examples/orders
# Compare two Goa design modules structurally at 0 LLM tokens
go run ./cmd/surla diff --old-dir examples/orders --new-dir examples/ordersHere examples/orders is the project and Surla runs a team of coding agents
on it. You need macOS, Go, Git, Docker (the builder checks its work in Linux
containers), Herdr (the terminal program the agents run in, one pane each),
Chorus v0.11.0 or newer (the service the agents message each other through,
whose dialog starts them) and a coding agent the Chorus dialog can start for
each role; the reference lists them
with their install commands. Verification also needs the Temporal CLI
(brew install temporal): the Orders evaluator suite starts a local Temporal
server for its scenarios. Install surla from this checkout first
(go install ./cmd/surla from the repository root).
In this project no test writer writes tests for each change. (In a project
without stored verification settings, a build also adds the test writer: an
agent that writes the tests the change requires in its own working copy.)
Here the tests are the evaluator suite in
examples/orders-eval, so the project
stores verification settings that name that suite at one commit. It also
stores promotion settings, which say where the binaries you will deploy are
published, so that approval also checks those binaries. Approval needs no
promotion settings; this project stores them to show that extra check. Both
files stay outside the project; the reference describes their fields under
surla verification and surla promotion.
From the repository root:
COMMIT="$(git rev-parse HEAD)"
WORK_DIR="$(mktemp -d)"
cat > "$WORK_DIR/verification.json" <<EOF
{
"candidate_repo": "$(pwd)",
"candidate_module": "examples/orders",
"binaries": {
"orders": "./cmd/orders",
"orders-worker": "./cmd/orders-worker"
},
"suite_repo": "$(pwd)",
"suite_commit": "$COMMIT",
"suite_module": "examples/orders-eval",
"suite_command": "./cmd/orders_acceptance-evals",
"timeout": "30m"
}
EOF
cat > "$WORK_DIR/promotion.json" <<EOF
{
"release_dir": "$WORK_DIR/releases",
"binaries": ["orders", "orders-worker"]
}
EOF
(
cd examples/orders
surla verification set "$WORK_DIR/verification.json"
surla promotion set "$WORK_DIR/promotion.json"
surla
)The subshell leaves your shell in the repository root for the next sections.
surla starts the project's Surla server in its own Herdr terminal, starts
Chorus and the Herdr app when they are not running, and opens the Chorus
dialog in Herdr, where you choose the agent app that plays the guide: the
agent you talk to, which writes the Story (a Go description of the business's
flows) and proposes changes. It prints Team <name>. and your next step,
opens the views page in your browser, then prints
Surla is running for <folder> at <address>. The server reads the stored
settings when it starts; after changing them, run surla stop, then surla,
in examples/orders.
One change then goes like this:
-
In the guide's pane, describe the change, for example "Card refunds above $100 need a second manager". The guide writes the design change in its own draft, the folder
.surla-draftthat Surla makes insideexamples/orders, never in your files. Then it proposes a Change Card: the saved record of the request, its open questions, and the examples that become checks. It appears on the views page. -
On the views page, answer the card's questions and confirm it. Only you can. The card's
request_revisionidentifies the revision you agreed. -
Press Build this change. Because this project has stored verification settings, the build adds only the builder, the agent that writes the code; no test writer is used. Choose its agent app in the Chorus dialog. The builder works in its own working copy of this repository: the folder
.surla-buildthat Surla makes insideexamples/orders, on the branchsurla/build-<n>, wherenis the build's number. Your checkout and the branch checked out in it stay as they are. -
The builder writes the change in its working copy, commits it on
surla/build-<n>and asks Surla to verify the commit (verify_change). Surla buildsordersandorders-workerfrom that commit and runs theorders_acceptancesuite at the stored commit against them. The result appears on the views page. -
Publish the binaries the run built, then approve. Because this project stores promotion settings, approval also checks the release folder. Put
ordersandorders-workerin$WORK_DIR/releases/<builder's commit>/, a folder named with that commit's full ID, writing the folder completely and renaming it into place in one step. Approval hashes those two files and accepts them only when their SHA-256 digests equal those of the binaries the run built. The run built them with its own build settings, so a build of your own need not match: publish the run's binaries. In the Proof view, select that completed run and copy its actual source folder from Binaries built by this run, shown as selectable code. Publishordersandorders-workerfrom that folder. Then press Approve this result. When approval refuses, the page shows Approval was not recorded. with the error and the selected run's binary folder, and nothing is saved. A missing folder or file, or a file that differs from the run's binary, is refused with the step "publish the binaries the run built, whose folder the run's Proof view shows, into the release folder that the promotion settings name, in a folder named after the commit, or remove the promotion settings". Publish them and press the button again, or remove the promotion settings (surla promotion remove, thensurla stopandsurla) to approve without that check.The approval names the builder's commit and the tests commit, here the stored suite commit. Surla merges and deploys nothing.
-
Merge the builder's commit, or drop the build. The builder's commit is on
surla/build-<n>, not on the branch checked out in your checkout of this repository. Once a run passed, the build shows Passed, and under To take this change, run in the project folder: the page gives the command to run inexamples/orders:git merge <candidate_commit>, the builder's commit. The tests are the stored suite's, so there is no tests commit to merge. When your branch holds that commit, the build shows Merged. To leave the change out instead, press Drop…, then Drop change: the builder's commits stay onsurla/build-<n>, and your branch is unchanged. Surla builds another change only once this build is merged or dropped.
Surla changes the refund agent's prompts only in a live target: a copy of
Orders it builds and runs itself, never an Orders process you started. Once a
build has added the builder, the builder calls start_live_target with the
confirmed change's change_id and a full candidate commit ID. This walkthrough
uses the stored Orders evaluator, so no accepted test-writer tests or chosen
model are needed. The server builds orders and orders-worker from that commit as
verification does, and the evaluator suite's --live mode starts them with
simulated commerce and payments, a Temporal dev server, and the evaluator's
scripted model. draft_override then sends new text for
orders.refund.system to the live target's prompts service with the key
Surla minted for it, and returns the accepted text's sha256: version; the
prompts service implements the
live target prompt contract.
A draft with no live target running is refused with no_live_target. The
views page's Live target view shows the running target and the prompts its
drafts changed. If it ends unexpectedly, the view retains its commit, launch
and observed-end times, and recovery message after confirming process cleanup.
draft_override then refuses with live_target_ended; no draft is sent.
Drafts are not verified and end with the live target. The refund agent's
prompt is part of the design, the SystemPrompt in design/orders.go, and
the builder never edits design/: generate_code writes the agreed design
there, and verify_change refuses a commit whose design/ files differ
from it. To keep a wording, the builder asks the guide to propose a revision
of the change with that wording. You adopt the revision and build it, and the
builder calls generate_code, commits, and verifies that commit.
The scripted model needs no provider key and answers from a fixed script, so a draft changes the prompt text and version the live target's model requests carry, not its replies. The live target's chat API needs bearer tokens signed with a key only the evaluator suite holds.
The same steps run as an automated test against the committed HEAD from
the repository root (it needs the temporal CLI and macOS):
go test -race -count=1 -tags e2e -run TestOrdersLiveCopyAppliesADraftThroughTheProjectServer ./cmd/surlasurla verify and surla approve are operator tools: they run the same
verification and approval check from a shell, without a server or a team.
surla verify checks out the exact committed CandidateCommit and SuiteCommit (uncommitted working-tree edits are never tested), builds orders and orders-worker into WorkDir/candidate-bin/, runs the orders_acceptance suite in examples/orders-eval (WorkDir/suite-bin/suite), and writes the cryptographically bound verify.Report JSON:
Scenario (orders_acceptance) |
Tag | What It Proves |
|---|---|---|
store_credit_within_limit |
repair |
Order totalling \$38.50 with tax is refunded as instant store credit without a manager (0 card payment writes). |
card_refund_needs_manager |
repair |
Order totalling \$54.00 including shipping pauses at Confirmation; customer and cross-tenant tokens cannot approve it (403), and that tenant's manager approval yields 1 card refund. |
refund_survives_worker_crash |
repair |
Killing orders-worker (SIGKILL / kill -9) right after the payment provider accepts a refund still yields 1 refund and a completed run on replay. |
pending_approval_survives_api_restart |
repair |
A refund waiting for a manager survives killing and restarting the orders API process; subsequent manager approval still yields 1 card refund. |
other_customers_order_refused |
holdout |
A customer asking to refund another customer's order is refused by server-bound tenant_id/customer_id lookup scoping (0 writes). |
Run the full evaluator report from the repository root:
COMMIT="$(git rev-parse HEAD)"
VERIFY_DIR="$(mktemp -d)"
# Full operator/evaluator view (includes holdout scenario details)
go run ./cmd/surla verify \
-candidate-repo . \
-candidate-commit "$COMMIT" \
-candidate-module examples/orders \
-binary orders=./cmd/orders \
-binary orders-worker=./cmd/orders-worker \
-suite-repo . \
-suite-commit "$COMMIT" \
-suite-module examples/orders-eval \
-suite-command ./cmd/orders_acceptance-evals \
-request-revision rev-orders-demo \
-work-dir "$VERIFY_DIR/work" \
-report-dir "$VERIFY_DIR/reports" \
-builder-view=falseThe builder-redacted report (Report.ForBuilders()) is the default. The explicit -builder-view below also requests it:
go run ./cmd/surla verify \
-candidate-repo . \
-candidate-commit "$COMMIT" \
-candidate-module examples/orders \
-binary orders=./cmd/orders \
-binary orders-worker=./cmd/orders-worker \
-suite-repo . \
-suite-commit "$COMMIT" \
-suite-module examples/orders-eval \
-suite-command ./cmd/orders_acceptance-evals \
-request-revision rev-orders-demo \
-work-dir "$VERIFY_DIR/work-builder" \
-report-dir "$VERIFY_DIR/reports-builder" \
-builder-viewBy default, stdout prints Report.ForBuilders() where the "holdout" scenario is redacted to "id": "holdout-1" with all scenario inputs, declarations, checks, and failure logs stripped. In either view, a run that stops with a passing partial report or no report still prints raw operator diagnostics to stderr; do not share those diagnostics with builder agents. (Note: in this public reference repo, examples/orders-eval is checked in alongside examples/orders to demonstrate the mechanism; in production, real hidden scenarios require keeping the evaluator suite repository (-suite-repo), -work-dir, and -report-dir unreadable to builder agents, since ForBuilders() redacts the report JSON while workspace boundaries protect the suite source and raw evidence on disk. The builder's programs run with your file access, so while a run executes they could read the suite's files on this machine.)
Once surla verify writes the full report to "$VERIFY_DIR/reports/<digest>.json", run surla approve (verify.Approve) to re-derive the report from its raw evidence and verify that the candidate commit, confirmed request_revision, evaluator suite, environment, and measured candidate binary SHA-256 digests match the change being promoted:
REPORT_FILE="$(ls "$VERIFY_DIR/reports/"*.json | head -n 1)"
ORDERS_SHA="$(shasum -a 256 "$VERIFY_DIR/work/candidate-bin/orders" | awk '{print $1}')"
WORKER_SHA="$(shasum -a 256 "$VERIFY_DIR/work/candidate-bin/orders-worker" | awk '{print $1}')"
go run ./cmd/surla approve \
-report "$REPORT_FILE" \
-candidate-commit "$COMMIT" \
-request-revision rev-orders-demo \
-suite-id orders_acceptance \
-suite-commit "$COMMIT" \
-environment sim \
-artifact "orders=$ORDERS_SHA" \
-artifact "orders-worker=$WORKER_SHA"Want to watch Surla reject a functional change that regresses LLM token efficiency? Run the acceptance suite directly:
(cd examples/orders-eval && go test -race -tags e2e -v ./acceptance)In addition to TestAcceptanceSuite, this runs TestSeededPayloadBloatFailsTheBudget, which rebuilds the candidate with a Go -overlay replacing ordersPageSize = 10 with ordersPageSize = 50 in internal/orders/service.go and proves that the evaluator's gRPC model gateway fails within_token_budget (input tokens jump from 2,563 to 4,193, exceeding refundBudget's 3,100 input-token / \$0.011200 cap in examples/orders-eval/acceptance/budget.go) even though every functional refund assertion still passes.