Skip to content

Commit c63c27d

Browse files
Juanpacolclaude
andcommitted
feat(experiments): arbitrary-tiebreak pilot -- the ceiling finally breaks
Five consecutive pilots (ADR-0011, 0015, 0016, 0017, 0019) ceilinged on task success. The closest attempt, pilot 7 (ADR-0019), used a genuinely code-unresolvable ambiguity (grace-period boundary inclusivity) but chose one with a dominant linguistic convention -- every trial guessed the fabricated policy correctly, unprompted, because "grace period" reads as inclusive by common usage. This pilot removes the convention entirely. Fixture: allocation/pick_winner.py's bug is a one-line no-op, trivial to spot -- not the point. The real question is a tie-break rule with zero linguistic signal: candidates carry only opaque numeric ids, and nothing in the naming suggests which one should win a tie in score. The hidden test (never shown to any trial) deliberately lists the lower-id (correct, per the fabricated policy) candidate SECOND in a tied pair, so the most natural-looking fix -- `max(candidates, key=lambda c: c["score"])` -- returns Python's first-seen maximum on a tie: the wrong candidate. Getting this right requires having actually considered that ties exist, which nothing visible prompts. Result: for the first time in nine Family B pilots, success itself splits. All 5 naive trials wrote the idiomatic max() one-liner -- reasonable, passes the visible test, silently wrong on a tie. All 5 verity trials read the handoff's explicit warning about max()'s tie behavior and wrote a tie-break comparison instead. tie_correct: naive 0/5 (floor [0.0, 0.0]), verity 5/5, likely_real_difference, zero overlap. The lesson, stated in ADR-0020: what determines whether recovery changes outcome rather than just cost is not task difficulty -- pilot 7's task was arguably harder to reason about -- it is the absence of any inferable signal in the ambiguity itself. "A lazy, idiomatic fix that passes the visible test and fails a hidden edge case nothing hints at" is now a reusable template, distinct from pilots 4-6's harder-tracing family and pilot 7's linguistically-loaded attempt. 520 tests passing (no source changes this phase). See docs/adr/0020-arbitrary-tiebreak-pilot.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 451992e commit c63c27d

13 files changed

Lines changed: 268 additions & 12 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,3 +98,4 @@ experiments/family_b_pilot_4_recovery_after_reset/trials/
9898
experiments/family_b_pilot_5_harder_recovery/trials/
9999
experiments/family_b_pilot_6_runtime_bug/trials/
100100
experiments/family_b_pilot_7_domain_ambiguity/trials/
101+
experiments/family_b_pilot_8_arbitrary_tiebreak/trials/

README.md

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Claude Code sessions of this project (~3.48M tokens,
7070
| 30,000-token budget, ranked against the task | 56.7% | 100% | 100% |
7171

7272
**Family B — a model decides something, so every claim carries a noise
73-
floor.** Eight pilots, reported whatever they found:
73+
floor.** Nine pilots, reported whatever they found:
7474

7575
| Pilot | Question | Verdict |
7676
|---|---|---|
@@ -82,14 +82,17 @@ floor.** Eight pilots, reported whatever they found:
8282
| [0017](docs/adr/0017-runtime-bug-pilot.md) | Does changing the bug's *shape* break it? | No. Cost effect reproduced a third time |
8383
| [0018](docs/adr/0018-consistency-engine-first-measurement.md) | Does the Consistency Engine catch real hallucinations? | 100% recall on invented symbols; three real bugs found and fixed |
8484
| [0019](docs/adr/0019-domain-ambiguity-pilot.md) | Does an ambiguity *not derivable from code at all* break the ceiling? | No — but for a new reason: the model's naming convention matched the policy in 10/10 trials regardless of condition |
85-
86-
The honest summary of those eight: **recovery after a context reset
87-
reliably makes an already-achievable outcome cheaper, and that is the most
88-
reproduced result here — but no design yet has shown it changes whether a
89-
capable agent succeeds at all.** Five consecutive ceilings across five bug
90-
designs — including one where the answer genuinely isn't in the code — is
91-
a finding about what these tasks can detect, and it is reported as such
92-
rather than buried.
85+
| [0020](docs/adr/0020-arbitrary-tiebreak-pilot.md) | Does an ambiguity with *no* linguistic convention finally break it? | **Yes.** `likely_real_difference` — 0/5 vs 5/5, the first success-rate split in the series |
86+
87+
The honest summary of those nine: **recovery after a context reset
88+
reliably makes an already-achievable outcome cheaper — the most reproduced
89+
result here — and it can also change whether the outcome is correct at
90+
all, when what's missing is knowledge no amount of reading the code can
91+
supply.** It took five consecutive ceilings and two attempts at a
92+
code-unresolvable ambiguity to isolate that condition precisely: not
93+
harder tracing, not vaguer wording, but an answer with no inferable signal
94+
anywhere in the repository. All nine results are reported as found, ceiling
95+
or not.
9396

9497
---
9598

docs/BENCHMARK_PROTOCOL.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,16 @@ independently — checking which one the fix actually implements. Fifth
5656
ceiling, 5/5 both conditions on the hidden check too, but for a new reason:
5757
every `naive` trial independently chose the interpretation matching the
5858
fabricated policy, with no access to it, because "grace period" carries a
59-
strong enough naming convention to override the genuine ambiguity. All
60-
seven pilots' stated limits (small N, narrow task design, single fixture)
61-
still apply — see each pilot's own README before generalizing any result.
59+
strong enough naming convention to override the genuine ambiguity. Family
60+
B, eighth pilot (`experiments/family_b_pilot_8_arbitrary_tiebreak/`,
61+
ADR-0020), corrected exactly that: an ambiguity with no linguistic
62+
convention at all (a tie-break rule between two opaque numeric IDs), with
63+
the hidden test's tied candidates ordered so the most natural-looking fix
64+
(`max(candidates, key=lambda c: c["score"])`) returns the wrong one on a
65+
tie. Result: `likely_real_difference`, 0/5 vs 5/5 -- the first success-rate
66+
split in the series, after five consecutive ceilings. All nine pilots'
67+
stated limits (small N, narrow task design, single fixture) still apply --
68+
see each pilot's own README before generalizing any result.
6269

6370
Separately, the Consistency Engine (`consistency/`, ADR-0007) got its own
6471
first real measurement (`experiments/consistency_pilot_1_hallucination_

docs/MEASUREMENTS.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,31 @@ a real technique for scoring "correct for the right reason" versus
235235
"happened to pass" — and proposes the next pilot use an ambiguity with no
236236
dominant convention (a coin-flip-shaped choice, not a language-shaped one).
237237

238+
**An eighth pilot found that ambiguity, and the ceiling broke.**
239+
(`experiments/family_b_pilot_8_arbitrary_tiebreak/`, [ADR-0020](adr/0020-arbitrary-tiebreak-pilot.md)):
240+
same hidden-test design, but the ambiguity is a tie-break rule between two
241+
opaque numeric IDs — nothing in the naming suggests an answer. The hidden
242+
test lists the correct (per the fabricated policy) tied candidate second
243+
in the input, so the most natural-looking fix
244+
(`max(candidates, key=lambda c: c["score"])`) returns the *wrong* one on a
245+
tie, because Python's `max` keeps the first-seen maximum:
246+
247+
| Metric | `naive` | `verity` | Verdict |
248+
|---|---|---|---|
249+
| Visible test passes (5 trials) | 5/5 | 5/5 | ceiling, as designed |
250+
| Correct on the hidden tie (5 trials) | 0/5 | 5/5 | `likely_real_difference` |
251+
252+
The first genuine success-rate split in eight Family B pilots. Every
253+
`naive` trial wrote the idiomatic `max()` one-liner — reasonable, passes
254+
the visible test, and wrong on a tie for a reason nothing visible reveals.
255+
Every `verity` trial read the handoff's explicit warning about `max()`'s
256+
tie behavior and wrote a tie-break comparison instead. The mechanism
257+
wasn't difficulty — pilot 7's task was arguably harder to reason about —
258+
it was the *absence of any inferable signal* in the ambiguity itself. See
259+
ADR-0020 for what made this design succeed where pilot 7's didn't, and its
260+
own stated limits (N=5, one fixture, a decision that named the rule
261+
explicitly).
262+
238263
**The Consistency Engine got its own first real measurement**
239264
(`experiments/consistency_pilot_1_hallucination_detection/`, [ADR-0018](adr/0018-consistency-engine-first-measurement.md)),
240265
closing a stale claim in `BENCHMARK_PROTOCOL.md` that this engine was
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# ADR-0020: The ceiling breaks — recovery changes outcome, not just cost
2+
3+
- **Status**: Accepted
4+
- **Date**: 2026-08-10
5+
- **Context**: pilot 7 (ADR-0019) tested a real code-unresolvable ambiguity
6+
(grace-period boundary inclusivity) but it carried a dominant linguistic
7+
convention strong enough that every trial guessed the fabricated policy
8+
correctly, unprompted. Five consecutive ceilings (ADR-0011, 0015, 0016,
9+
0017, 0019) raised the question of whether any design in this family
10+
could show recovery changing an outcome, not just its cost.
11+
12+
## Decision: an ambiguity with no linguistic tell, and a hidden test that exposes the laziest wrong fix
13+
14+
`allocation/pick_winner.py`'s bug is a one-line no-op — trivial, not the
15+
point. The real question: on a tie in `score`, which of two candidates
16+
wins? Candidates carry only opaque numeric `id`s; nothing in the naming
17+
suggests an answer, unlike "grace period." The fabricated phase-A decision
18+
states the policy (lower `id` wins) explicitly and warns that `max()`'s
19+
default tie behavior does not reliably match it.
20+
21+
The hidden test (never shown to any trial) uses two tied candidates with
22+
the lower-`id` one listed **second** in the input — deliberately, so that
23+
`max(candidates, key=lambda c: c["score"])`, the most natural-looking
24+
correct fix, returns Python's first-seen maximum on a tie: the wrong
25+
candidate here. This exposes the exact failure mode recovery is supposed
26+
to prevent — a fix that looks entirely correct, passes the shown test, and
27+
is wrong for a reason nothing visible reveals.
28+
29+
## Result
30+
31+
| Metric | naive | verity | Verdict |
32+
|---|---|---|---|
33+
| `visible_pass` (5 trials) | 5/5 | 5/5 | ceiling, as designed |
34+
| `tie_correct` (5 trials) | 0/5 | 5/5 | `likely_real_difference` |
35+
36+
All 5 `naive` trials wrote the idiomatic `max()` one-liner, with no way of
37+
knowing it was wrong on a tie — nothing in the code or the visible test
38+
says ties are even possible. All 5 `verity` trials read the handoff,
39+
encountered the explicit warning about `max()`'s tie behavior, and wrote a
40+
tie-break comparison instead. Zero overlap between the noise floors.
41+
42+
## Consequences
43+
44+
- **This is the finding the series was built to test for.** Five ceilings
45+
established that recovery reliably lowers cost on tasks a model already
46+
solves; this pilot shows recovery can also change whether the task is
47+
solved *correctly*, when the thing being recovered is knowledge — not
48+
code structure — that no amount of reading the repository can supply.
49+
- **The mechanism was the ambiguity's shape, not its difficulty.** Pilot 7
50+
failed to show a split with a harder-seeming but linguistically-loaded
51+
ambiguity; this pilot succeeded with an easier-seeming but genuinely
52+
unknowable one. The lesson for future pilots in this family: difficulty
53+
is not what determines whether recovery changes outcomes — the absence
54+
of any inferable signal is.
55+
- **The design generalizes.** "A lazy, idiomatic fix that passes the
56+
visible test and fails a hidden edge case nothing hints at" is a
57+
reusable template for testing recovery in this project going forward,
58+
distinct from pilots 4-6's "harder to trace" family and pilot 7's
59+
"linguistically ambiguous" attempt.
60+
- Same discipline as every prior pilot: verified by hand, before spending
61+
trial budget, that the lazy fix fails the hidden case and the
62+
tie-break-aware fix passes both, and checked for giveaway text in the
63+
fixture.

docs/adr/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ verdict was "this pilot could not have detected an effect."
3737
| [0017](0017-runtime-bug-pilot.md) | Does changing the bug's *shape* break it? | No — fourth ceiling; cost effect reproduced a third time |
3838
| [0018](0018-consistency-engine-first-measurement.md) | Does the Consistency Engine catch real hallucinations? | 100% recall on invented symbols; three real bugs found and fixed |
3939
| [0019](0019-domain-ambiguity-pilot.md) | Does an ambiguity not derivable from code at all break the ceiling? | No — the model's naming convention matched the fabricated policy 10/10 regardless of condition |
40+
| [0020](0020-arbitrary-tiebreak-pilot.md) | Does an ambiguity with no linguistic convention finally break it? | **Yes**`likely_real_difference`, 0/5 vs 5/5, the first success-rate split in the series |
4041

4142
## Pre-pivot (superseded)
4243

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Arbitrary-tiebreak pilot: the ceiling finally breaks
2+
3+
Per `docs/BENCHMARK_PROTOCOL.md`'s Family B procedure. Pilots 4-7 (ADR-0015,
4+
0016, 0017, 0019) all ceilinged on success — even pilot 7's genuinely
5+
code-unresolvable ambiguity produced 10/10 identical answers, because the
6+
specific ambiguity chosen ("grace period") carried a strong linguistic
7+
convention. This pilot removes that convention entirely.
8+
9+
**Status: complete, and this is the first pilot in the series where
10+
success itself splits between conditions.** 5/5 `naive` trials get the
11+
answer wrong; 5/5 `verity` trials get it right. `likely_real_difference`.
12+
13+
## The design
14+
15+
`fixture_repo/allocation/pick_winner.py`'s bug is a one-line no-op (always
16+
returns the first candidate) — trivial to spot, not the point. The real
17+
question: when two candidates tie on `score`, which one wins? Nothing in
18+
the domain suggests an answer — candidates are identified by opaque
19+
numeric `id`s, with no suggestive naming anywhere (unlike pilot 7's "grace
20+
period," which reads as inclusive by convention).
21+
22+
The visible test uses candidates with clearly different scores, so any
23+
correct-looking comparison passes it regardless of tie-break behavior. The
24+
hidden test (never shown to any trial) uses two candidates **tied on
25+
score**, with the lower-`id` candidate listed **second** in the input —
26+
deliberately, because `max(candidates, key=lambda c: c["score"])`, the
27+
laziest fix that still looks entirely correct, returns Python's
28+
first-seen maximum on a tie: the higher-`id` candidate here, which is
29+
wrong per the fabricated policy ("on a tie, the lower id wins"). Getting
30+
this right requires having actually considered that ties exist, not just
31+
writing a plausible one-liner.
32+
33+
## Phase A, conditions, scoring
34+
35+
Same shape as pilots 4-7 (`../lib/setup_phase_a.sh`): `naive` gets the bare
36+
task and no `.verity/`; `verity` gets a `.verity/` pre-loaded with the
37+
fabricated decision, which explicitly states the tie-break rule and warns
38+
that `max()`'s default behavior does not reliably match it. 5 live-agent
39+
trials per condition, Bash access, constrained not to touch `tests/`.
40+
41+
Two metrics, both scored independently by the harness:
42+
- `visible_pass`: does the shown test pass? (expected ~10/10, a sanity
43+
check, not the finding)
44+
- `tie_correct`: does `pick_winner` return the lower-`id` candidate on the
45+
hidden tie case? This is the metric that matters.
46+
47+
## Result
48+
49+
| Metric | naive | verity | Verdict |
50+
|---|---|---|---|
51+
| `visible_pass` (5 trials) | 5/5 | 5/5 | ceiling, as designed |
52+
| `tie_correct` (5 trials) | 0/5 | 5/5 | `likely_real_difference` |
53+
54+
All 5 `naive` trials wrote `max(candidates, key=lambda c: c["score"])`
55+
a genuinely reasonable, idiomatic fix that passes the visible test
56+
perfectly and that none of them had any way of knowing was wrong on a tie.
57+
All 5 `verity` trials read the handoff, saw the explicit warning about
58+
`max()`'s tie behavior, and wrote a tie-break comparison instead. Noise
59+
floor `[0.0, 0.0]` for `naive`, between-config mean `1.0` for `verity`
60+
zero overlap. Reproduce:
61+
62+
```bash
63+
./setup_phase_a.sh
64+
# run each of the 10 trials as a live agent per the design above
65+
verity noise-floor naive_results.json verity_results.json --metric tie_correct
66+
```
67+
68+
## Why this pilot succeeded where pilot 7 didn't
69+
70+
Pilot 7's ambiguity (grace-period boundary inclusivity) had a dominant
71+
natural-language convention strong enough that agents guessed the correct
72+
policy without ever being told it. This pilot's ambiguity has no such
73+
convention: nothing about numeric `id`s or a "highest score wins" rule
74+
suggests which one wins a tie. The result is the first genuine
75+
success-rate split in eight Family B pilots — not because the task got
76+
harder in a way that a longer investigation would close, but because the
77+
correct answer was never inferable from the code or its naming at all,
78+
and here that was actually true rather than only apparently true.
79+
80+
## Known limitations of this pilot, stated up front
81+
82+
- **One fixture, one arbitrary rule, 5 trials per condition.** Same
83+
caveat as every prior pilot — see `docs/BENCHMARK_PROTOCOL.md` before
84+
generalizing to "recovery always changes outcome on domain ambiguities."
85+
It took two attempts (pilot 7, then this one) to find an ambiguity that
86+
actually worked; a third ambiguity might again have a hidden convention.
87+
- **The `verity` decision was quite explicit** (it names the exact tie-break
88+
rule and warns against the specific wrong default). This is consistent
89+
with every prior pilot's phase-A fabrication, but it means this result
90+
shows recovery helps when the recovered decision directly answers the
91+
ambiguity — not that recovery helps with vaguer or less complete notes.
92+
- **`visible_pass` and `tie_correct` are perfectly correlated with
93+
condition here** (ceiling on one, clean split on the other) — a larger N
94+
would be needed to characterize how consistently this replicates, not
95+
just whether it can happen at all.

experiments/family_b_pilot_8_arbitrary_tiebreak/fixture_repo/allocation/__init__.py

Whitespace-only changes.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""Winner selection for a limited allocation slot."""
2+
3+
4+
def pick_winner(candidates: list[dict]) -> dict:
5+
return candidates[0]
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import sys
2+
from pathlib import Path
3+
4+
sys.path.insert(0, str(Path(__file__).parent.parent))
5+
6+
from allocation.pick_winner import pick_winner
7+
8+
9+
def test_highest_score_wins():
10+
candidates = [
11+
{"id": 1, "score": 10},
12+
{"id": 2, "score": 50},
13+
]
14+
assert pick_winner(candidates)["id"] == 2

0 commit comments

Comments
 (0)