reconcile.py reads any input a person brings and grades a supplied statement part by part

[sandeep-agami]

Spec: ACE-115 (feature reconcile-evidence, brief F10)

Stacked on #288. Base is ACE-114-reconcile-sm-verbs, because the ledger reads what those four verbs emit. Merge #288 first, then retarget this to main. Do not merge without manual review.

Summary

agami-reconcile is learning to take the evidence a person brings, in any of four shapes, and to grade a supplied SQL statement part by part instead of trusting it. #288 added the verbs that describe a statement; this PR adds the deterministic half that reads the input and grades the parts. Nothing here runs SQL, and nothing writes to the semantic model. The skill change that wires it all together is the next PR.

Three verbs in plugins/agami/scripts/reconcile.py, beside parse, diff and band, which are unchanged and still pinned by tests/test_reconcile.py:

• intake --file <path> [--file …] [--source …] reads a list of questions, questions with the SQL the person trusts, labels with numbers, or labels with numbers and the SQL behind each tile, into one row shape {question?, statement?, expected?} plus provenance. Detection is by content, never by asking. A CSV whose third column is SQL yields a statement rather than a label with SQL glued onto it; a third column that is not SQL is still context, as parse always read it. Several files merge by label under a case and whitespace fold. Exit 2 for a file it cannot open, 4 for an input with nothing usable.
• ledger --row-dir <dir> [--with-claims] grades every part of a supplied statement from fixed filenames in its row directory: what happened when it ran, what sm prepare and sm receipt said, what sm join-probes and sm filter-values judge reported, and the probe CSVs the execution tier returned. Four grades (confirmed, model_gap, query_defect, unresolved), and only measurement earns model_gap. The dependency rule is applied: a join that could not be graded leaves the fan-out check on its aggregate unresolved, said out loud. A scope refusal is a finding about the semantic model, not a crash. A zero-byte probe CSV is a probe that failed. The weakest part decides the verdict; the counts travel with it.
• findings --run-dir <dir> writes findings.json (one entry per problem, the same missing join seen from two statements in either order counted once, and a filter gap never colliding with a metric gap), query_defects.json (the person's own defects, listed apart so nothing about the semantic model is proposed from them), and ledger.json. A clean statement the AI answered differently is a finding of kind example.

Three shared references carry the procedure the skill and agami-save-correction will both follow: shared/evidence-row.md, shared/part-ledger.md, and shared/statement-check.md, the last saying how a supplied statement runs the road the AI's own SQL runs, by reference to agami-query, never with weaker guards. shared/file-layout.md gains the run directory.

Verification

• tests/test_reconcile_intake.py: 15 tests, the four shapes, the pinned three-column label case, JSON input, header recognition by name, merging by label, provenance, skips and exit codes.
• tests/test_reconcile_ledger.py: 30 tests, one per grading rule (the run, joins in every status, the probe rows, aggregates and the dependency rule, filters, metrics with the bare-count exemption, typed values, claims), precedence, idempotence, and the findings' keys, kinds and defects list.
• tests/test_reconcile.py passes untouched (parse, diff, band byte-identical).
• uv run dev.py check: ruff and gitleaks clean, no vendored-lib drift, 5642 passed, 12 skipped, 1 pre-existing failure: tests/test_golden_run.py::test_a_client_that_cannot_be_found_still_fails_as_a_generation, which fails identically on untouched origin/main on this machine because a client binary exists at /opt/homebrew/bin/claude. Not touched by this change.

Review round 2

A second panel pass, fixed in the second commit with regression tests (ledger 45, intake 17):

• Missing inputs are open parts. After a successful run, an absent, zero-byte, errored, or wrong-shaped prepare, receipt, join-probes or judge file becomes fan_out:* / receipt:* / join:* / literal:* graded unresolved, naming the file and the problem. Grading only what was there made a crashed verb read as a clean statement.
• undetermined output columns are unresolved, not model_gap; a failure to read is never a gap. The matched metric's evidence reads the receipt's name key.
• Case-insensitive CSV headers, and the one-column fall-back never reads a wider row (uppercase headers flipped a unique key into a repeating one).
• One failed overlap probe beside a zero is unresolved, not "keys never meet". Two joins between the same tables are two parts (#2). A join without a status stays open.
• An example finding needs every part confirmed and a row that was graded at all. Filter finding keys drop the alias, so one declared filter is one finding.
• A .sql file's chunks get the statement check, so a DROP never becomes a row. Exit 4 still prints what was skipped and why.
• Run evidence carries the classifier's kind and remediation, never stderr.
• statement-check.md: the profile root on every verb (three commands were missing it), statements and probes passed by file path rather than inline in a shell string, and a .run.json record beside every probe so every execution and refusal in the phase is written down.

Review round 3

From a test run on a real profile. Third commit, ledger tests 54 passing:

• A count through declared many-to-one joins is confirmed. The pre-flight leaves COUNT(*) undetermined because it names no column; when every join the statement writes is listed, confirmed, and brings in one row at most (its right side is the declared relationship's one side, or its written column is unique by the semantic model), nothing can multiply the count. The rule reads every join part, because the pre-flight's own joins list is empty for exactly this aggregate.
• The open note names the blindness. "the pre-flight could not bind this aggregate to one table: ", with the reason sm prepare now carries, instead of blaming a join where nothing was wrong.
• A fifth word, noted, for a fact the run states and never judges. It never decides the row's verdict and never blocks an example. First use: dropped_rows:<a>-<b>, the left table's rows with no partner on the right, from the probe Four sm verbs grade a statement a person supplied, part by part #288 now plans.
• values_declared:<table>.<column>, one per filtered column: model_gap when the column holds a short list of values the semantic model lists none of, noted for a wide, empty or sensitive column, unresolved when the distinct probe did not run. Same finding family as a stale list, so it reaches findings.json.
• A metric matched by shape on a table the statement never reads is unresolved, using the source_tables the receipt now carries.

Review round 4

Fourth panel pass on the third commit. Fixed in the fourth, with regression tests (ledger tests 68 passing):

• The one-row stamp read the statement's lowercased key against the semantic model's own spelling, so on an uppercase-introspected model every declared join read as the many side and no count through it ever reached the keep-offer. Both sides are folded now. The fallback's live shape is tested; the earlier test used a shape the verb cannot emit.
• The dropped-rows part reads both numbers by header, never the one-column fall-back that would state N of N dropped as a fact, and names the side the probe did not count.
• Tests for the empty and failed distinct states, a sensitive column that is also listed, and a metric over several tables read through one of them.

🤖 Generated with Claude Code