Skip to content

Commit 10c7f73

Browse files
docs: replace the inherited baseline with a measured one
The 31.6s in the hand-off notes did not survive re-measurement. Against the fork point, kernel built from each commit's own source and confirmed active on both sides: 41.74s -> 26.22s serial, 40.02s -> 25.81s at each side's default. Neither side's parallelism is worth much; the win is the batching. Also records that the old process pool died with KeyError outside the __main__ entry point on any spawn-default platform, which removing it fixes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 1b5305b commit 10c7f73

1 file changed

Lines changed: 22 additions & 6 deletions

File tree

docs/superpowers/plans/2026-08-15-phase-b-batch-prediction.md

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1586,12 +1586,28 @@ Then report, best of 3 each: serial batch vs threaded batch vs the 31.6s pre-Pha
15861586
The plan was executed and these are the results, kept here because the hand-off notes this plan was
15871587
written from are deleted by the last task.
15881588

1589-
**The serial win is the payoff.** 20-scenario benchmark, best of 3: **26.3s** against the 31.6s
1590-
recorded before this work (that baseline was inherited, not re-measured on the same machine). It
1591-
comes from paying the ctypes boundary and the window marshalling once per fan-out.
1592-
1593-
**Threading buys nothing measurable: 26.27s serial vs 26.05s at 16 threads**, inside the 2-6%
1594-
run-to-run noise band. Results are bit-identical either way.
1589+
**The win is the batching itself, and it is large.** Measured against `a131f12c`, the fork point,
1590+
with the kernel built from each commit's own source and confirmed active on both sides, runs
1591+
interleaved, best of 3, on a 16-CPU machine. All 20 scenarios byte-identical in every configuration.
1592+
1593+
| 20-scenario benchmark | fork point | this branch | delta |
1594+
|---|---|---|---|
1595+
| parallelism off (`threads: 0`) | 41.74s | 26.22s | **37.2% faster** |
1596+
| each side's default (`threads: auto`) | 40.02s | 25.81s | **35.5% faster** |
1597+
1598+
The 31.6s figure in the hand-off notes was measured in an earlier session under unknown conditions
1599+
and did not survive re-measurement — do not carry it forward.
1600+
1601+
**Neither side's parallelism is worth much.** The fork point's 16-process pool bought 41.74s → 40.02s
1602+
(4%); this branch's 16 kernel threads buy 26.22s → 25.81s (1.6%), inside the 2-6% noise band. The
1603+
entire improvement is the batching and the marshalling it removes, not concurrency.
1604+
1605+
**The fork point's process pool was also broken outside the `__main__` entry point.** Its workers
1606+
rebuild their `Prediction` from a module global populated only in the parent, which requires `fork`
1607+
start semantics; `hass.py` sets those only under `if __name__ == "__main__"`. On macOS (spawn by
1608+
default since 3.8) every worker therefore started empty and every scenario died with
1609+
`KeyError: 'dict'` — the benchmark above needed `fork` forced to get comparable numbers out of the
1610+
baseline at all. This branch removes that failure mode by removing the pool.
15951611

15961612
**Why**, from a `n_jobs` census over one benchmark run:
15971613

0 commit comments

Comments
 (0)