Skip to content

docs: report on the planning performance work - #4547

Merged
springfall2008 merged 1 commit into
mainfrom
docs/planning-performance-report
Aug 16, 2026
Merged

docs: report on the planning performance work#4547
springfall2008 merged 1 commit into
mainfrom
docs/planning-performance-report

Conversation

@springfall2008

Copy link
Copy Markdown
Owner

Documentation only — no code changes.

The report on the planning performance work was written while #4540 and #4546 were still open, and landed on the branch just after each was merged, so it missed both. This carries it in on its own.

It records, for the work now in main:

  • The measured result — 39.53s to 24.82s on the 20-scenario benchmark, 37% faster, plans byte-identical. Measured against main at the time, each side on its own Python and kernel, interleaved, best of three.
  • What changed — the batched fan-out, and the kernel thread pool that replaced 201,545 thread creations per plan.
  • Four assumptions a benchmark overturned — scratch reuse that turned out to be a no-op, a broadcast wake that cost most of the win, a thread cap that was right on fast hardware and 10.7% wrong on slow, and an inherited baseline that did not survive re-measurement.
  • The defects found on the way — the ABI version that let a stale binary load and segfault instead of falling back, the old process pool being broken off the __main__ entry point, and a header that only libc++ provides transitively.
  • How the tests were checked — five tests, three of them new, that could not have failed until they were mutation-checked.

Filed under docs/superpowers/ with the plans and specs rather than in the MkDocs nav, since it is an engineering record rather than user documentation.

🤖 Generated with Claude Code

Records the measured result (39.5s to 24.8s on the 20-scenario benchmark, plans unchanged), what the
two PRs changed, and the four design assumptions that a benchmark overturned - the scratch reuse that
turned out to be a no-op, the broadcast wake that cost most of the win, the thread cap that was right
on fast hardware and wrong on slow, and an inherited baseline that did not survive re-measurement.

Also records the defects found on the way, including the ABI version that let a stale binary load and
segfault, and the five tests that could not have failed until they were mutation-checked.

Lives under docs/superpowers/ with the plans and specs rather than in the MkDocs nav: it is an
engineering record, not user documentation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 16, 2026 15:39
@springfall2008
springfall2008 merged commit 87cb016 into main Aug 16, 2026
3 checks passed
@springfall2008
springfall2008 deleted the docs/planning-performance-report branch August 16, 2026 15:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an engineering report documenting the planning-performance improvements that landed via PR #4540 and #4546, capturing benchmark methodology, measured results, key design changes, and notable defects/assumptions uncovered during the work.

Changes:

  • Introduces a detailed performance report (benchmark results, methodology, and analysis) for the batching and kernel thread-pool work.
  • Records measurement-driven conclusions (what helped, what didn’t, and why) plus defects found and how tests were validated.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants