Skip to content

Commit 995e05a

Browse files
committed
merge: M3-W117 -- candidate 2, and the chained-call defect handed on
2 parents 9bc6fa5 + e62acfd commit 995e05a

1 file changed

Lines changed: 144 additions & 1 deletion

File tree

docs/superpowers/reports/2026-07-30-the-corpus-scores-one-kind.md

Lines changed: 144 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,126 @@ So the answer to "would a partial match arise in a pinned repository" is yes, an
162162
to be fabricated to get one. The blocker is entirely on the generator side. That is the argument
163163
for candidate 3 and it is why this report names it as the next task rather than as a dead end.
164164

165+
## The three candidates weighed, and the two rejected
166+
167+
**Taken: candidate 2.** `request-parameter-removed` inverts, and inverts to a pair the corpus
168+
already holds. The finding is the identity, and a test pins it so the claim cannot rot into
169+
folklore. No corpus specification is added and the recorded score is untouched, so it is still the
170+
frozen ruler the verification regime relies on and still byte-identical to what was recorded.
171+
172+
Worth naming plainly: `scripts/build_corpus_specs.py`'s own docstring already asserts this —
173+
"the third supported kind, `request-parameter-removed`, mutates identically to the first and would
174+
add pairs without adding information." So candidate 2 does not contradict the codebase, it
175+
*measures* a claim the codebase made about itself and had never checked. That is the whole
176+
contribution and it should not be dressed up as more.
177+
178+
**Rejected: candidate 1**, adding `request-parameter-removed` specifications. Rejected on its own
179+
measurement rather than on preference. Five added pairs are five exact duplicates of their property
180+
twin, and the effect on the published number is to move `n=26` to `n=35` while every added row is a
181+
row already counted. That is the number getting better because the corpus got easier, which
182+
`2026-07-29-sync-ground-truth-quality.md` forbids, and it is worse here than the usual form of that
183+
mistake: the inflation is invisible in the score, since both rates stay at 1.0000 and only the
184+
denominator moves. A reader has no way to see it from the artifact.
185+
186+
It also costs the corpus its stated invariant. `benchmark/corpus/README.md` holds the directory to
187+
"every operation the rule proposes is in the corpus", with the corpus a deliberate superset — five
188+
extra today. The rule does not propose the parameter kind at all, so candidate 1's five
189+
specifications would all be hand-written, widening the superset gap from five to ten with rows that
190+
carry no evidence. Paying 27 seconds a run for that is the smallest of the objections.
191+
192+
**Rejected: candidate 3**, widening the generator to express a nested change — but rejected as
193+
*out of scope here*, not as wrong. It is the answer with the real evidence behind it: the flat
194+
corpus genuinely cannot produce a false positive, five of the seven reachable negatives in the
195+
pinned checkouts would give a partial path match something to fire on, and no repository has to be
196+
fabricated. What rules it out for this task is that it changes the ruler. `changed_field` taking
197+
the leaf is what makes every committed pair's tree what it is, so widening it regenerates the
198+
frozen corpus, and the brief's own constraint — a measurement task first, do not manufacture a
199+
diff — puts a change of that size behind its own argument rather than inside this one. Candidate 3
200+
is the next task on the corpus axis and the section above is the argument for it.
201+
202+
## A separate and larger finding: the mutation attaches to the wrong call on a chained site
203+
204+
This is not part of the candidate question and is not fixed here. It surfaced while measuring
205+
`request-parameter-removed` and it is reported as its own next task below.
206+
207+
`_call_at` takes an exact start match and then breaks a tie by preferring a call that passes an
208+
object argument and, failing that, **the longest one**. A chained call and its receiver start at
209+
the same line and column, so both match, and the longest is the outer call. Enumerated directly at
210+
the position the indexer records:
211+
212+
```
213+
=== python: position line=1 col=22, kind='call' ===
214+
matches at that exact start position: 2
215+
len= 61 object_argument=False 'client.coupons.list(params={"limit": 100}).auto_paging_iter()'
216+
len= 42 object_argument=False 'client.coupons.list(params={"limit": 100})'
217+
_call_at takes: 'client.coupons.list(params={"limit": 100}).auto_paging_iter()'
218+
219+
=== typescript: position line=1 col=19, kind='call_expression' ===
220+
matches at that exact start position: 2
221+
len= 65 object_argument=True 'stripe.charges.list({ limit: 3 }).autoPagingToArray({ limit: 5 })'
222+
len= 33 object_argument=True 'stripe.charges.list({ limit: 3 })'
223+
_call_at takes: 'stripe.charges.list({ limit: 3 }).autoPagingToArray({ limit: 5 })'
224+
```
225+
226+
**The object-argument preference is inert on a chained call in both languages, for opposite
227+
reasons.** In Python neither call has an object argument — `params={...}` is a `keyword_argument`,
228+
not a `dictionary` child of `arguments`, and the pager takes nothing. In TypeScript both do, because
229+
a pager taking its own options is how `autoPagingToArray` is written. Either way the first key does
230+
not discriminate and length decides. So it is not a partial mitigation that chained calls slip past;
231+
it never applies to them at all.
232+
233+
The consequence is a mislabel. The field the vendor removed from the *operation* is written into
234+
the *pager's* argument list, and the label still says the site is affected. Removing `created` from
235+
`GET /v1/coupons` cannot break `auto_paging_iter(created=…)`, so `affected=True` describes a break
236+
the tree does not carry, and a binder that correctly declines is scored as having missed it. That
237+
understates recall with a manufactured miss — the same failure mode the nested-change section above
238+
records, arriving through a different door.
239+
240+
**The audit half cannot catch it.** `depends_on_change` exists so a label can be checked against
241+
what the tree says rather than against what the generator recorded, and it is the one guard that
242+
could have flagged this. It resolves the position through the same `_call_at`, so it asks about the
243+
pager too and truthfully answers that the pager carries the field. A cross-check strong enough to
244+
catch this has to resolve the position the way the indexer does, and `mutate.py` deliberately
245+
imports nothing from `sync.index` — so the fix belongs in the tie-break, not in a second opinion.
246+
247+
### Why it is latent, and how far from live
248+
249+
No committed specification names an operation whose call sites chain, which is why the corpus has
250+
never scored a mislabelled pair. The margin is thinner than that sounds. Across the five pinned
251+
checkouts there are thirteen chained pager sites — nine in `virtual-lab`, four in `furever` — and
252+
`virtual-lab`'s chained calls name five distinct list operations: `coupons`, `customers`,
253+
`invoices`, `prices`, `subscriptions`.
254+
255+
The decisive detail is in the corpus itself. `virtual-lab-GetProductsId-response-property-removed.yaml`
256+
records in its own header that the rule already proposed two of these operations and they were
257+
refused: "The two candidate operations B45 proposed both bound through `list(...auto_paging_iter())`
258+
and were refused for it." They were refused on the **response** axis, for the result-binding a
259+
response guard needs — not for the misattachment, which nobody had noticed. The request axis
260+
imposes no such requirement: `_insert_property` needs an object argument or, in Python, falls back
261+
to a keyword insertion, and a chained `list(params={...})` satisfies that. **So a
262+
`request-property-removed` pair on any of those five operations is one specification away, would be
263+
accepted by the rule that generated the seventeen, and would land straight on this defect.** The
264+
operations were already proposed once; only a response-side condition sent them back.
265+
266+
### What the next task owns
267+
268+
Fixing `_call_at`'s tie-break so it prefers the call the indexer recorded, and regenerating the
269+
frozen corpus in the same commit — because changing which call is mutated changes every pair's
270+
tree, and a recorded score taken against the old attachment stops describing the generator that
271+
produced it.
272+
273+
Two measurements the next task starts from rather than re-deriving. Inverting the length tie-break
274+
kills exactly the two chained assertions and leaves both unchained controls green, so the fix
275+
direction does not regress the ordinary single-call site. And the audit-half tests pass under that
276+
inversion too, which is the evidence that `depends_on_change` is not a check on this and would not
277+
have to change.
278+
279+
It is deliberately not done here. `mutate.py` is mine to edit where a test proves a defect, and a
280+
test does — but this fix moves the published number, and moving it as a side effect of a
281+
measurement task is the thing the brief's constraint exists to prevent. It is also a second reason
282+
to reach for `_call_at` alongside candidate 3, and the two should be weighed together rather than
283+
landed separately, since both regenerate the corpus.
284+
165285
## Every assertion in the pinning test was proved able to fail
166286

167287
`tests/test_mutation_kind_coverage.py` pins behaviour that already worked, so "watch it fail
@@ -187,7 +307,30 @@ compile, nothing exited outside `{0,1}`, no pass count drifted, and no anchor mi
187307
CRLF normalisation in the harness is there to prevent, since this tree is CRLF and an anchor
188308
written with `\n` matches nothing and reads as a survival.
189309

190-
Scheduler: `-n0` (serial) for every pytest run in the harness, which is also what makes the
310+
`tests/test_mutation_call_selection.py`, which pins the chained-call misattachment, was put through
311+
the same harness. Five mutants, baseline `exit=0 passed=6 failed=0` before and after restore, and
312+
the failing node ids collected per mutant rather than only the counts — a count cannot show that
313+
*every* test was covered, only that some were.
314+
315+
| mutant | verdict | detail | aimed at |
316+
|---|---|---|---|
317+
| `tie-break-prefers-shortest` | killed | 2 failed | the chained tie-break takes the outer call |
318+
| `python-keyword-insert-declines` | killed | 1 failed | the Python mutation attaches, and to which call |
319+
| `ts-property-insert-declines` | killed | 3 failed | the TypeScript mutation attaches, and to which call |
320+
| `audit-half-ignores-keywords` | killed | 1 failed | the audit half agrees, via the Python keyword branch |
321+
| `audit-half-object-branch-declines` | killed | 3 failed | the audit half agrees, via the object-literal branch |
322+
323+
Six of six distinct tests proved able to fail, and again no false-verdict mode. Every anchor was
324+
checked for being unique in the file before it was applied, so a `not-applied` verdict from an
325+
ambiguous target could not be misread as a survival.
326+
327+
The first mutant is the informative one twice over. It kills the two chained assertions and
328+
**nothing else** — both unchained controls and both audit-half tests stay green — which is what
329+
makes those two tests a statement about chaining rather than about the parameter kind or about
330+
these particular call shapes, and which is also the measurement that says the fix direction does
331+
not regress the ordinary site.
332+
333+
Scheduler: `-n0` (serial) for every pytest run in both harnesses, which is also what makes the
191334
baseline `12 passed in 3.13s` rather than the 20.41s a `-n auto` run of the same file costs in
192335
worker startup. The pipeline runs are `scripts`-level and single-process. Score DSN
193336
`sync_w117_score`, graph DSN `sync_w117`, both created by this task.

0 commit comments

Comments
 (0)