reconcile: the card reads agreement, says could not compare, names the failed side (ACE-134) - #320
Open
sandeep-agami wants to merge 1 commit into
Open
Conversation
…e failed side; 2e unordered
One differing cell printed "0% of the values match" for a column that was there; the values row now
reads the comparator's share ("9 of 10 rows match", "differs in total on 1 of 10 rows"), and "same
rows, different columns" needs the paired columns to agree on every row. Every error row said
"agami's query failed" although the cause was often the person's statement, two empty results or a
bare question; the card reads the cause from the row's files, offers agami again only when agami
failed, and sends a question-only row to the grading page. The result pill reads "could not
compare", or "same query, answer not compared" / "different query, answer not compared" when the
claims could be read, the `outputs` claim ("selects") included, so query equivalence stands on its
own when the data cannot be compared. The near-miss sentence shows only on a defect. Phase 2e passes
`--unordered`: a different sort is a different query, never a different answer.
Pins moved on purpose: "identical, row for row" reads "identical"; the error sentence carries its
cause; "could not run" reads "could not compare"; the 2e pin names --unordered.
Spec: ACE-134
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec: ACE-134 (agami-sdlc
projects/agami-core-extension/specs/reconcile-evidence/ACE-134-reconcile-card-agreement.md)Stacked on #319 (ACE-133); reads the score fields and the
outputsclaim from #316 (ACE-131) and falls back without them. Round 9 of the reconcile stack, from the 51-row run.The findings
errorrow said "agami's query failed" in the label, the sentence and the change text, althougherrormeans the row could not be scored: 18 error rows, agami's statement had run fine on 11. Phase 3 may rewritesentenceandchange, neverresult.label.("could_not_compare", *)always read "could not run".--golden-sql-file, so row order decided the comparison (ACE-131's finding A).The change (
plugins/agami/scripts/reconcile.py, the report template,SKILL.md2e,part-ledger.md)accuracy1.0 reads "identical"; with pairs and apaired_row_share, 1.0 reads "identical on the N paired columns" and below 1.0 reads "K of N rows match" with the note "differs in total on 1 of 10 rows" per pair whosecolumn_agreementis below 1. An older score file keeps today's branches._values_agree_on_shared_columnsreads the share, so "partly" needs the paired columns to agree on every row._error_cause, from the files:yours_failed(therunsorscopepart not confirmed),agami_failed(no statement, or nothing recorded with an error),nothing_to_compare(the scoreunscored),no_ground_truth(a question-only row), elseunknown. The label is "could not compare"; the sentence "This row could not be compared: : "; the fixask_againonly foragami_failedandunknown("run this row again"),queryorsemantic_modelforyours_failed,nonewith its own change text for the other two; the owner follows.outputsclaim reads "selects" and is definitional. Labels for a row whose data could not be compared: "same query, answer not compared" (amber), "different query, answer not compared" (red), "could not compare" (gray). The keep gate is untouched:status == matchfrom the run.defect(ACE-130's other half).SKILL.md2e:--unorderedreplaces--golden-sql-file; the eight claims named.part-ledger.md: which claims are parts and which the page reads;orderingis the one place row order is judged.Pins moved on purpose
"identical, row for row"→"identical"(two); the error sentence"agami's query failed: boom."→"This row could not be compared: agami's query failed: boom."; the label"could not run"→"could not compare"; the 2e pin--golden-sql-file→--unordered. The plan's note about the end-to-end test's compare helper does not apply: it diffs scalars.Tests
New: one differing cell reads "9 of 10 rows match" with the differing column named; paired columns agreeing on every row beside an unpaired one read "partly", and not when they disagree too; an older score without the share keeps the grace; each of the six error causes yields its fix, owner, sentence and change text; an error row whose claims all agree reads "same query, answer not compared" and a differing
outputsalone reads "different query, answer not compared" with a "selects" row, and is never keepable; a different projection alone is a different query. 259 passed across the reconcile suites, the promotion-skill pins and the end-to-end test.The gallery artifact is republished after the remaining page changes in this round (ACE-135, ACE-136).
🤖 Generated with Claude Code