Skip to content

Commit 70354be

Browse files
eval: prove native Agent Spec resources
agent-session-id: dev3.dotfiles-cos-misc-st2-resource-design agent-tool: Codex agent-tool-version: 0.145.0 agent-model: gpt-5.6-sol agent-runtime-profile: /home/schickling/.local/state/agent-session-recovery/2026-07-29-pty-st2-cutover/runtime-profile/profile-without-null-opencode.json agent-skills-manifest: /nix/store/kx5j47nghj1yps2v693ryb6wnf1c2xhb-agent-skills-corpus/share/agent-skills/manifest.json tooling-profile: dotfiles@16a22c6
1 parent 58f2d11 commit 70354be

16 files changed

Lines changed: 312 additions & 23 deletions

File tree

AGENT-SPEC.md

Lines changed: 52 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Canonical st2 agent specification
22

33
This is the sole agent-authoring specification for this repository. It is pinned to st2
4-
[`9887b2842222def0838c2cd82e6c24c218f7efa6`](https://github.com/compoundingtech/st2/commit/9887b2842222def0838c2cd82e6c24c218f7efa6)
5-
(`0.1.0`, source `9887b28`). It documents the hand-authored KDL accepted at that commit. Do not infer
4+
[`0fed14bb5653b67e1d64f1199e240c4c5c612bf7`](https://github.com/compoundingtech/st2/commit/0fed14bb5653b67e1d64f1199e240c4c5c612bf7)
5+
(`0.1.0+0fed14b`, source `0fed14b`). It documents the hand-authored KDL accepted at that commit. Do not infer
66
additional fields or commands from older corpus fixtures.
77

88
st2 runs long-lived `service` agents made of interactive `pty` tasks and terminal-free `exec` tasks.
@@ -46,6 +46,8 @@ agent "<identity>" {
4646
retired #false
4747
keep #false
4848
49+
resource "work" _tag="github-issue" uri="github-issue://example/project/123"
50+
4951
restart {
5052
attempts 3
5153
interval "60s"
@@ -82,6 +84,7 @@ Supported agent children are:
8284
| `supervisor "…"` | Optional bare identity or full bus id for crash-loop routing. |
8385
| `retired #true` | Decommission the declaration on the next reconciliation. Edit this flag; do not delete a live declaration to retire it. |
8486
| `keep #true` | Freeze dead evidence and suppress collection/restart for every task; retirement still stops live tasks. |
87+
| `resource "name" _tag="type" uri="absolute-uri"` | Binds one uniquely named, externally identified Resource as declaration metadata. |
8588
| `restart { … }` | Optional service restart policy. |
8689
| `env { KEY "value" }` | Environment inherited by the compact agent task and sidecars. |
8790
| `command "…"` | Compact interactive task named `agent`. |
@@ -100,6 +103,41 @@ the window reset; `mode "fail"` parks the task after attempts are exhausted and
100103
crash-loop message to `supervisor`. Invalid restart subfields currently fall back to defaults; authors must not
101104
rely on that permissiveness.
102105

106+
## Resource bindings
107+
108+
An agent may directly carry zero or more Resource bindings:
109+
110+
```kdl
111+
resource "work" _tag="github-issue" uri="github-issue://example/project/123"
112+
resource "source" _tag="worktree" uri="worktree://example/project/main"
113+
```
114+
115+
The positional name is the Resource's agent-local semantic role. Names are non-empty and unique within one
116+
agent. `_tag` is a non-empty, opaque discriminator owned by the Resource type's downstream contract. `uri` is
117+
an RFC 3986 absolute URI and is the Resource identity. st2 preserves the URI's exact bytes; it does not
118+
normalize or resolve it. Declaration order has no meaning. Canonical KDL and supported TOML/JSON parsing lower
119+
bindings to deterministic name order.
120+
121+
The generic envelope is closed: each binding has exactly the positional name, `_tag`, and `uri`. Missing or
122+
duplicate fields, duplicate names, child nodes, invalid URI syntax, and unsupported properties such as access
123+
or readiness policy fail validation. This prevents an ignored property from appearing enforced.
124+
125+
Resource bindings are declaration metadata, not launch targets. They do not make an otherwise unrunnable
126+
service runnable and are excluded from effective task launch definitions. Editing only Resource bindings
127+
therefore updates catalog inspection while an already-live task is adopted without stop, replacement, or
128+
relaunch. `st2 agents --json [--enrich]` exposes every binding as a name-ordered
129+
`{"name","_tag","uri"}` descriptor and preserves unknown downstream tags.
130+
131+
The envelope does not define Resource schemas, resolution, access grants, required/optional status, readiness,
132+
lifecycle, mutation, or rendering. A URI's presence grants no authority. Those semantics belong to the
133+
concrete Resource type and its consumer, not st2.
134+
135+
Executable evidence:
136+
[`agent-spec-resource-bindings`](cells/agent-spec-resource-bindings/) covers strict parser failures,
137+
deterministic JSON inspection, exact URI and unknown-tag preservation, Resource-only live adoption, and
138+
cleanup. The matched [`assignment-contract-*`](cells/) tournament covers direct Resource selection against
139+
Focus and Assignment controls; direct bindings are the selected treatment.
140+
103141
## Compact and explicit tasks
104142

105143
The canonical compact pair:
@@ -392,7 +430,7 @@ Codex launch explicitly selects `gpt-5.6-sol` at medium reasoning effort.
392430

393431
The folder-eval grammar preserves its `team`, `eval`, kickoff, timeout, and held-out judge shape. Its
394432
agent projection supports `workspace`, `supervisor`, `env`, `command`, explicit `exec`, and bare
395-
`ding`; it does not accept a catalog `render` block. For that grammar, `eval { copy … }` and a
433+
`ding`; it does not accept catalog `resource` or `render` nodes. For that grammar, `eval { copy … }` and a
396434
deterministic pre-boot materializer must place the equivalent harness files in each declared
397435
workspace:
398436

@@ -415,9 +453,9 @@ agent { command "true" }
415453
At the pinned source it validates as one agent with zero errors and warnings. Production declarations should
416454
normally make identity, host, `ST_AGENT`, workspace, and the real harness command explicit.
417455

418-
The complete declaration, compact pair, explicit PTY/exec blocks, restart policy, and render block earlier in
419-
this document collectively exercise every implemented authoring field. Before validating/materializing the
420-
example, create every `$CATALOG`-rooted workspace it names.
456+
The complete declaration, Resource bindings, compact pair, explicit PTY/exec blocks, restart policy, and
457+
render block earlier in this document collectively exercise every implemented authoring field. Before
458+
validating/materializing the example, create every `$CATALOG`-rooted workspace it names.
421459

422460
## Free authoring gate
423461

@@ -432,8 +470,11 @@ Inspect the declaration, every referenced template, and every workspace destinat
432470
materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting
433471
the network is a separate, explicitly authorized action.
434472

435-
For source `9887b28`, the accepted Linux executable has SHA256
436-
`d49d44fd4f3f6f655455c212353a469fefa956082bedf22163deb767d8a36a0d`; its published archive has SHA256
437-
`32ee103bd17ccb3e155ac63d816a3906c2470a3c98e3cc04b56e5a67138b9927`. `bin/check-corpus.sh` verifies
438-
the variable-age version contract, exact installed binary, embedded full source commit, strict semantic
439-
validation, fixture resets, and the rest of the model-free corpus gate before an eval may run.
473+
For source `0fed14b`, the accepted pre-merge Nix-built Linux executable has SHA256
474+
`96b394f270f0a3eb25dd29574a96f30d527a56bee63c2499d6db7e4a58707648`. There is no published portable
475+
archive for this unmerged commit. After `compoundingtech/st2#86` merges, this pin must move to its accepted
476+
release and record the fresh-download archive hash before the corpus is ready. `bin/check-corpus.sh` verifies
477+
the variable-age Nix version contract, exact installed candidate binary, strict semantic validation, fixture
478+
resets, and the rest of the model-free corpus gate before an eval may run. The pre-merge Nix stamp embeds only
479+
the short revision, so the full source link above is review/build provenance rather than a claim extracted
480+
from the binary; the post-merge portable release must restore fresh-download full-source proof.

CATALOG.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<!-- Generated by bin/generate-catalog.sh from cell KDL and evidence/*.tsv. Do not edit. -->
22
# Current eval catalog and evidence
33

4-
This is the sole current corpus surface: **53 included cells** (13 model-free,
4+
This is the sole current corpus surface: **54 included cells** (14 model-free,
55
40 model-backed) and **4 retired exclusions**. Inventory fields are derived from the active
66
KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run
77
history: a failed last run is visible without being advertised as accepted evidence, and cells with no
@@ -16,6 +16,7 @@ harness-native loader plus canonical hook file.
1616

1717
| Cell | Harness | Model(s) / effort | Model seats | Cost | Timeout | Held-out judges | Latest accepted PASS | Last recorded run |
1818
|---|---|---|---:|---|---|---:|---|---|
19+
| `agent-spec-resource-bindings` | model-free || 0 | none | `60s` | 4 || **NO STRUCTURED RUN** |
1920
| `assignment-contract-cold-assignment` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 || **NO STRUCTURED RUN** |
2021
| `assignment-contract-cold-focus` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 || **NO STRUCTURED RUN** |
2122
| `assignment-contract-cold-resources` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 || **NO STRUCTURED RUN** |
@@ -89,6 +90,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
8990

9091
## Harness hook exclusions
9192

93+
- `agent-spec-resource-bindings` / `cell` — deterministic native Agent Spec Resource-envelope probe; no harness seat
9294
- `context-resource-continuity` / `cell` — deterministic native context/resource restart probe; no harness seat
9395
- `docs` / `judge:cold-reader` — one-shot offline Claude print grader; no bus identity, DING, or hook surface
9496
- `exec-lifecycle-logging` / `cell` — deterministic native exec lifecycle probe; no harness seat
@@ -104,7 +106,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
104106
- `st2-network` / `cell` — deterministic catalog/message round-trip probe; no harness seat
105107
- `two-networks-coexist` / `cell` — deterministic catalog-isolation probe; no harness seat
106108

107-
These **14** rows are generated from structured exclusions. A cell-level row is required for
109+
These **15** rows are generated from structured exclusions. A cell-level row is required for
108110
every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected
109111
agent: it has no st2 identity, DING sidecar, or lifecycle hooks.
110112

README.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ VERDICT: PASS
9696
```
9797

9898
Requirements are `st2 0.1.0` from source
99-
[`9887b28`](https://github.com/compoundingtech/st2/commit/9887b2842222def0838c2cd82e6c24c218f7efa6),
99+
[`0fed14b`](https://github.com/compoundingtech/st2/commit/0fed14bb5653b67e1d64f1199e240c4c5c612bf7),
100100
`pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell
101101
also needs every harness named by its dry-run row.
102102

@@ -122,6 +122,17 @@ current bus-connected model agent.
122122
Team-less cells use deterministic `run` steps and judges without a model. Current examples cover native hook
123123
materialization, network health, catalog/pty isolation, and pty send/peek behavior.
124124

125+
## Native Resource-envelope acceptance
126+
127+
`agent-spec-resource-bindings` is the model-free companion to
128+
[`compoundingtech/st2#86`](https://github.com/compoundingtech/st2/pull/86). It exercises native catalog
129+
declarations rather than the tournament's synthetic Resource documents. The cell proves strict envelope
130+
validation, deterministic `st2 agents --json` inspection, opaque downstream tags, exact URI preservation, and
131+
adoption of a live task after a Resource-only declaration edit.
132+
133+
Folder-eval Resource projection, Resource resolution, access, readiness, and lifecycle policy are outside this
134+
cell. The portable Agent Spec envelope does not imply any of them.
135+
125136
## Resource-binding tournament
126137

127138
The nine `assignment-contract-*` cells form one matched tournament over three lifecycle scenarios and three

bin/check-corpus.sh

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,9 @@ set -euo pipefail
55
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
66
cd "$repo_root"
77

8-
expected_source="9887b28"
9-
expected_source_full="9887b2842222def0838c2cd82e6c24c218f7efa6"
10-
expected_binary_sha256="d49d44fd4f3f6f655455c212353a469fefa956082bedf22163deb767d8a36a0d"
11-
expected_version_regex='^st2 0\.1\.0 — running from local source \(9887b28, .+ ago\)$'
8+
expected_source="0fed14b"
9+
expected_binary_sha256="96b394f270f0a3eb25dd29574a96f30d527a56bee63c2499d6db7e4a58707648"
10+
expected_version_regex='^st2 0\.1\.0\+0fed14b — committed .+ ago$'
1211
st2_path="$(command -v st2)"
1312
actual_version="$(st2 --version)"
1413
[[ "$actual_version" =~ $expected_version_regex ]] || {
@@ -20,11 +19,7 @@ actual_binary_sha256="$(sha256sum "$st2_path" | awk '{ print $1 }')"
2019
echo "FAIL: expected st2 binary sha256 $expected_binary_sha256, found $actual_binary_sha256 at $st2_path" >&2
2120
exit 1
2221
}
23-
LC_ALL=C grep -aFq "$expected_source_full" "$st2_path" || {
24-
echo "FAIL: st2 binary at $st2_path does not embed full pinned source $expected_source_full" >&2
25-
exit 1
26-
}
27-
echo "PASS: pinned runner source $expected_source ($actual_version; sha256 $actual_binary_sha256)"
22+
echo "PASS: pinned pre-merge Nix candidate $expected_source ($actual_version; sha256 $actual_binary_sha256)"
2823

2924
mapfile -d '' shell_files < <(
3025
find bin cells -type f -name '*.sh' -not -path '*/_git/*' -print0 | sort -z
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# agent-spec-resource-bindings
2+
3+
Model-free acceptance coverage for the native Agent Spec Resource envelope introduced by
4+
[`compoundingtech/st2#86`](https://github.com/compoundingtech/st2/pull/86).
5+
6+
The cell proves the catalog contract directly through st2:
7+
8+
- Resource names are unique, `_tag` remains opaque, and `uri` must be an RFC 3986 absolute URI.
9+
- Unsupported policy properties fail closed instead of being silently ignored.
10+
- Canonical KDL and supported TOML/JSON forms project the same stable, name-ordered
11+
`st2 agents --json` descriptors without normalizing URI bytes.
12+
- Editing only Resource bindings updates the declared roster while the existing PTY keeps the same process
13+
identity and is adopted rather than stopped or relaunched.
14+
15+
It does not exercise folder-eval Resource projection, resolution, access, readiness, or Resource lifecycle
16+
policy. Those are outside the portable envelope.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
// Native Agent Spec Resource envelope: parse failures, roster projection, and nondisruptive adoption.
2+
eval {
3+
copy "./fixture"
4+
max-timeout "60s"
5+
6+
run "contract" {
7+
command "bash ./contract.sh"
8+
}
9+
10+
judges {
11+
judge "STRICT ENVELOPE — malformed URI, duplicate names, and policy fields fail closed" {
12+
exec "grep -Fqx RESOURCE-STRICT-FAILURES-GREEN-c214 $RUNS_DIR/contract.out"
13+
}
14+
judge "INSPECTION — agents JSON is deterministic and preserves opaque tags plus exact URI bytes" {
15+
exec "grep -Fqx RESOURCE-INSPECTION-GREEN-c214 $RUNS_DIR/contract.out"
16+
}
17+
judge "ADOPTION — a Resource-only declaration edit updates inspection without replacing the live task" {
18+
exec "grep -Fqx RESOURCE-NONDISRUPTIVE-ADOPTION-GREEN-c214 $RUNS_DIR/contract.out"
19+
}
20+
judge "CLEANUP — the catalog leaves no live or retained PTY state" {
21+
exec "grep -Fqx RESOURCE-CLEANUP-GREEN-c214 $RUNS_DIR/contract.out"
22+
}
23+
}
24+
}
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
root="${CATALOG:?CATALOG must be set}"
5+
net="$root/net"
6+
spec="$net/agents/rb/worker/agent.kdl"
7+
export PTY_ROOT="$net/pty"
8+
export XDG_STATE_HOME="$root/state"
9+
10+
cleanup() {
11+
if test -f "$spec"; then
12+
sed -i '/role "worker"/a\\ retired #true' "$spec" 2>/dev/null || true
13+
st2 up --once --catalog "$net" --host rb >/dev/null 2>&1 || true
14+
PTY_ROOT="$PTY_ROOT" pty rm rb.worker >/dev/null 2>&1 || true
15+
fi
16+
}
17+
trap cleanup EXIT
18+
19+
for case_name in uri duplicate policy; do
20+
output="$root/invalid-$case_name.out"
21+
set +e
22+
st2 validate --catalog "$root/invalid/$case_name" --host rb --strict >"$output" 2>&1
23+
exit_code="$?"
24+
set -e
25+
test "$exit_code" -ne 0
26+
done
27+
grep -Fq 'must be an exact absolute URI' "$root/invalid-uri.out"
28+
grep -Fq "duplicate resource binding 'work'" "$root/invalid-duplicate.out"
29+
grep -Fq 'unsupported property `access`' "$root/invalid-policy.out"
30+
echo "RESOURCE-STRICT-FAILURES-GREEN-c214"
31+
32+
st2 validate --catalog "$root/parity" --host rb --strict >/dev/null
33+
st2 agents --catalog "$root/parity" --host rb --json >"$root/parity.json"
34+
jq -e '
35+
length == 3 and
36+
[.[].identity] == ["rb.json", "rb.kdl", "rb.toml"] and
37+
([.[].resources] | unique | length) == 1 and
38+
.[0].resources == [
39+
{
40+
"name": "context",
41+
"_tag": "vendor-specific-v7",
42+
"uri": "vendor+thing://authority/context%2Fone"
43+
},
44+
{
45+
"name": "work",
46+
"_tag": "github-issue",
47+
"uri": "github-issue://example/project/41?view=exact%20bytes"
48+
}
49+
]
50+
' "$root/parity.json" >/dev/null
51+
52+
st2 validate --catalog "$net" --host rb --strict >/dev/null
53+
st2 agents --catalog "$net" --host rb --json >"$root/agents-before.json"
54+
st2 agents --catalog "$net" --host rb --json >"$root/agents-repeat.json"
55+
cmp "$root/agents-before.json" "$root/agents-repeat.json"
56+
jq -e '
57+
length == 1 and
58+
.[0].identity == "rb.worker" and
59+
.[0].resources == [
60+
{
61+
"name": "context",
62+
"_tag": "vendor-specific-v7",
63+
"uri": "vendor+thing://authority/context%2Fone"
64+
},
65+
{
66+
"name": "work",
67+
"_tag": "github-issue",
68+
"uri": "github-issue://example/project/41?view=exact%20bytes"
69+
}
70+
]
71+
' "$root/agents-before.json" >/dev/null
72+
echo "RESOURCE-INSPECTION-GREEN-c214"
73+
74+
st2 up --once --catalog "$net" --host rb >"$root/launch.out"
75+
grep -Fq 'launched (1): rb.worker' "$root/launch.out"
76+
before="$(
77+
PTY_ROOT="$PTY_ROOT" pty list --json |
78+
jq -cer '.[] | select(.name == "rb.worker" and .status == "running") | {name, pid, createdAt}'
79+
)"
80+
test -n "$before"
81+
82+
sed -i \
83+
's#resource "work" _tag="github-issue" uri="github-issue://example/project/41?view=exact%20bytes"#resource "work" _tag="github-pr" uri="github-pr://example/project/86?view=exact%20bytes"#' \
84+
"$spec"
85+
sed -i \
86+
'/resource "context"/a\\ resource "terminal" _tag="pty" uri="pty://rb/rb.worker"' \
87+
"$spec"
88+
st2 validate --catalog "$net" --host rb --strict >/dev/null
89+
st2 up --once --catalog "$net" --host rb >"$root/adopt.out"
90+
grep -Fq 'adopted (1): worker' "$root/adopt.out"
91+
test -z "$(sed -n '/launched (/p;/torn down (/p' "$root/adopt.out")"
92+
after="$(
93+
PTY_ROOT="$PTY_ROOT" pty list --json |
94+
jq -cer '.[] | select(.name == "rb.worker" and .status == "running") | {name, pid, createdAt}'
95+
)"
96+
test "$after" = "$before"
97+
98+
st2 agents --catalog "$net" --host rb --json >"$root/agents-after.json"
99+
jq -e '
100+
.[0].resources == [
101+
{
102+
"name": "context",
103+
"_tag": "vendor-specific-v7",
104+
"uri": "vendor+thing://authority/context%2Fone"
105+
},
106+
{
107+
"name": "terminal",
108+
"_tag": "pty",
109+
"uri": "pty://rb/rb.worker"
110+
},
111+
{
112+
"name": "work",
113+
"_tag": "github-pr",
114+
"uri": "github-pr://example/project/86?view=exact%20bytes"
115+
}
116+
]
117+
' "$root/agents-after.json" >/dev/null
118+
echo "RESOURCE-NONDISRUPTIVE-ADOPTION-GREEN-c214"
119+
120+
sed -i '/role "worker"/a\\ retired #true' "$spec"
121+
st2 up --once --catalog "$net" --host rb >/dev/null
122+
PTY_ROOT="$PTY_ROOT" pty rm rb.worker >/dev/null 2>&1 || true
123+
trap - EXIT
124+
test "$(PTY_ROOT="$PTY_ROOT" pty list --json | jq 'length')" -eq 0
125+
echo "RESOURCE-CLEANUP-GREEN-c214"
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
agent "worker" {
2+
host "rb"
3+
workspace "$CATALOG"
4+
resource "work" _tag="github-issue" uri="github-issue://example/project/41"
5+
resource "work" _tag="github-pr" uri="github-pr://example/project/86"
6+
command "true"
7+
}
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
agent "worker" {
2+
host "rb"
3+
workspace "$CATALOG"
4+
resource "work" _tag="github-issue" uri="github-issue://example/project/41" access="write"
5+
command "true"
6+
}
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
agent "worker" {
2+
host "rb"
3+
workspace "$CATALOG"
4+
resource "work" _tag="github-issue" uri="not-an-absolute-uri"
5+
command "true"
6+
}

0 commit comments

Comments
 (0)