Knowing how coding agents fail, and being able to catch it with checks the agent never sees, is a prerequisite for trusting them with real work.
This release candidate packages a small study of coding-agent failure modes. It contains ten task specifications, three executable examples, two six-cell OpenCode matrices, a failure taxonomy and a strict evidence trail.
The primary matrix recorded six transport-valid invocations, five completed tasks and two technical passes across public and hidden checks. A supplemental run produced the same aggregate counts with different cell outcomes. The case study treats that variation as a reason to collect more trials.
CASE_STUDY.md: methods, results and failure autopsies.EVAL_CARD.md: intended use, limitations and validity threats.DISCLOSURE_AND_HOLDOUT_POLICY.md: publication boundary for tests and traces.evidence/: redacted run cards, summaries and trace excerpts in an exported bundle.
python3 scripts/validate_tasks.pyExpected output:
OK: 10 tasks validated
The public bundle treats the three v0 fixtures as disclosed examples. It excludes hidden test source, raw grader output, local paths, session identifiers and generated cache files.
v0.1.0-rc1 release candidate
The three executable fixtures are disclosed examples in this candidate, not unseen holdouts. Do not use this release to claim a model ranking or general failure rate.
This bundle is produced by an exporter that ships with the private source workspace and is not part of the public release. It refuses to overwrite an existing directory and runs a private-path and session-ID scan before it writes the hash manifest.