A comparison edge can carry its own date - #439
Open
ccerv1 wants to merge 2 commits into
Open
Conversation
A capability band placed against a peer is dated today by the peer's whole-axis last_verified. Those are two claims about two different products, and as the corpus grows every new product's natural peer was confirmed before the product existed, so the comparison graph stops growing at the periphery. capability.comparison dates the edge itself and carries the read of the peer that dated it. An attested edge is freed from the peer's axis date and pays for it with its own evidence requirement: a source read on or after last_attested, carrying http_status and content_sha256. Where the peer has been re-read since the spacing was judged, the edge is reported for re-judgment rather than failed. An edge with no attestation is unchanged, so the 181 recorded today keep the old rule and nothing is retroactively legitimized. check_refetch now walks the new source list, and the attestation date rides in axis_assessments' existing basis_detail string rather than a new column, which a deployed static model cannot take.
evidence-and-freshness.md gains the normative split between the date on a product's own capability evidence and the date on a spacing, plus the rule on what a digest match may prove and what it may never prove. capability.md documents the field and the gate. add-product.md gains the attest-the-spine step and lists check_capability, which CI has always run and the workflow never named. refresh-category.md keeps anchor-first mandatory and points at the alternative.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
check_capabilitybinds a dependent's whole-axiscapability.last_verifiedto its root'swhole-axis
capability.last_verified. Those are two claims about two different products, and thecoupling has a consequence #436 measured: as the corpus grows, every new product's natural root
was confirmed before the product existed, so a tranche can compare its members to each other and
to nothing else. It happened three times in one week — #431, #432 and #433 each dropped
arithmetically correct comparisons into prose rather than assert a re-derivation nobody performed.
181 of 553 products sit in the comparison graph today; 305 have no edge in or out.
The mechanism
An optional
capability.comparisonblock dates the edge rather than either product, andcarries the read that dated it:
Its own source list, not
capability.sources, because these are citations about somebody else'sproduct: folding them in would let a root's page count as this product's own evidence and would
make the weak-root check read the wrong thing. The shape is
#/definitions/sourceverbatim, sobuild/fetch_source.pywrites it unchanged andbuild/check_refetch.pyre-checks it onceload_sourceswalks the extra list — which it now does.Gate semantics
comparisonblockcapability.last_verifiedmust exist and be>=the dependent's.comparisonpresent, dependent'slast_verified>last_attestedrelative_to/relationare part of the axis's score, so a whole-axis confirmation cannot outrun one of its parts.comparisonpresent,last_attested> every attestation source'saccessedcheck_verification.invariant.accessed >= last_attested) lackinghttp_statusorcontent_sha256check_verification.digests.comparisonpresent,last_attested< the root'scapability.last_verifiedvaluemoved and its score did not.comparisonpresent and consistent, root's axis date older than the dependent'scheck_capability's summary gains two lines: how many recorded comparisons are attested versusresting on their root's axis date, and the re-judge queue from row five.
Why hash-only reaffirmation was rejected
Option B in the issue — let a comparison stand when the root's cited sources still return the same
content_sha256— fails on two counts.It is unsound. A spacing can go false with no byte changing anywhere, because the falsifier is a
third product that neither side cites. And it does not reach the roots that matter. Re-fetching
every digest-carrying capability source on all 54 comparison roots: 22 of 84 sources reproduce,
12 of 54 roots have every capability source reproduce, and of the 13 roots the three dropped
tranches needed, zero reproduce. The reason is structural — 26 of the 95 sources on those roots
are GitHub landing pages carrying a star count and 13 more are READMEs on active repos, while the
stable ones are arXiv abstracts and dataset cards. So the cheap path exists and does not go where
the expansion needs to go.
An attestation therefore costs a real read of the root. It is cheaper than a full anchor refresh in
what it claims, not in what it costs.
docs/reference/evidence-and-freshness.mdnow states themay-prove / may-never-prove rule so it does not drift back.
Migration
Mechanism only, no backfill. All 181 existing edges carry no
comparisonblock and stay on the oldrule, which they satisfy today. Backfilling would mean attesting 181 spacings, which is the work
this issue exists to make affordable rather than a precondition for it.
Anchor-first stays mandatory in
refresh-category— a refresh pass is where a re-read that moves ascore belongs, and it is the read that catches a root whose surface changed under the bands resting
on it (
openhands). Attestation is whatadd-productuses so a batch is not blocked betweenrefresh cycles.
Deliberately out of v1
last_attested. The 30-day freshness window is about axes; adding an edgedimension to it is its own decision, and the report line surfaces the same information without a
new failure mode.
check_capabilitywould accept a reciprocalatpair today, which isarithmetically consistent and semantically circular. Separate issue.
details_payload'sAXIS_KEYStrimscomparisonout, and itstays trimmed. Surfacing "spacing last judged on" in the product panel is a front-end question,
and there is no payload size gate forcing the call now.
The attestation date rides inside
axis_assessments' existingbasis_detailstring(
relative_to=X;relation=Y;attested=DATE) rather than a new column, because a column add on adeployed static model fails on the platform.
Scores
No score changes. This PR adds a mechanism and writes no comparison edges, so no product's
score, no category's
L, and no stage moves.build/notebook_data.jsonregenerates byte-identical.Validation
Closes #436