11<!-- Generated by bin/generate-catalog.sh from cell KDL and evidence/*.tsv. Do not edit. -->
22# Current eval catalog and evidence
33
4- This is the sole current corpus surface: ** 47 included cells** (16 model-free,
5- 31 model-backed) and ** 4 retired exclusions** . Inventory fields are derived from the active
4+ This is the sole current corpus surface: ** 57 included cells** (17 model-free,
5+ 40 model-backed) and ** 4 retired exclusions** . Inventory fields are derived from the active
66KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run
77history: a failed last run is visible without being advertised as accepted evidence, and cells with no
88structured run row say so explicitly.
@@ -17,6 +17,16 @@ harness-native loader plus canonical hook file.
1717| Cell | Harness | Model(s) / effort | Model seats | Cost | Timeout | Held-out judges | Latest accepted PASS | Last recorded run |
1818| ---| ---| ---| ---:| ---| ---| ---:| ---| ---|
1919| ` adopt-only-migration ` | model-free | — | 0 | none | ` 90s ` | 6 | — | ** NO STRUCTURED RUN** |
20+ | ` agent-spec-resource-bindings ` | model-free | — | 0 | none | ` 60s ` | 4 | — | ** NO STRUCTURED RUN** |
21+ | ` assignment-contract-cold-assignment ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 3 | — | ** NO STRUCTURED RUN** |
22+ | ` assignment-contract-cold-focus ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 3 | — | ** NO STRUCTURED RUN** |
23+ | ` assignment-contract-cold-resources ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 3 | — | ** NO STRUCTURED RUN** |
24+ | ` assignment-contract-handoff-assignment ` | Codex | ` gpt-5.6-sol ` / medium | 3 | high | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
25+ | ` assignment-contract-handoff-focus ` | Codex | ` gpt-5.6-sol ` / medium | 3 | high | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
26+ | ` assignment-contract-handoff-resources ` | Codex | ` gpt-5.6-sol ` / medium | 3 | high | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
27+ | ` assignment-contract-hot-assignment ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
28+ | ` assignment-contract-hot-focus ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
29+ | ` assignment-contract-hot-resources ` | Codex | ` gpt-5.6-sol ` / medium | 2 | medium | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
2030| ` context-resource-continuity ` | model-free | — | 0 | none | ` 90s ` | 4 | — | ** NO STRUCTURED RUN** |
2131| ` crash-ding ` | mixed | ` claude-sonnet-5+gpt-5.6-sol ` / medium | 4 | high | ` 180s ` | 5 | — | ** NO STRUCTURED RUN** |
2232| ` ding-mode ` | Claude | ` claude-sonnet-5 ` / medium | 2 | medium | ` 1200s ` | 4 | — | ** NO STRUCTURED RUN** |
@@ -84,6 +94,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
8494## Harness hook exclusions
8595
8696- ` adopt-only-migration ` / ` cell ` — deterministic native adoption/replacement lifecycle probe; no harness seat
97+ - ` agent-spec-resource-bindings ` / ` cell ` — deterministic native Agent Spec Resource-envelope probe; no harness seat
8798- ` context-resource-continuity ` / ` cell ` — deterministic native context/resource restart probe; no harness seat
8899- ` docs ` / ` judge:cold-reader ` — one-shot offline Claude print grader; no bus identity, DING, or hook surface
89100- ` exec-lifecycle-logging ` / ` cell ` — deterministic native exec lifecycle probe; no harness seat
@@ -101,7 +112,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
101112- ` targeted-reconcile-isolation ` / ` cell ` — deterministic selected reconcile isolation probe; no bus-connected agent
102113- ` two-networks-coexist ` / ` cell ` — deterministic catalog-isolation probe; no harness seat
103114
104- These ** 17 ** rows are generated from structured exclusions. A cell-level row is required for
115+ These ** 18 ** rows are generated from structured exclusions. A cell-level row is required for
105116every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected
106117agent: it has no st2 identity, DING sidecar, or lifecycle hooks.
107118
0 commit comments