Commit 14873d6
committed
docs(readme): verified before/after benchmarks for all 5 opt-in wrappers
Measured 2026-04-22, N=3 per data-point, fresh state dir per cold
column, against fixtures where sensible and real artifacts otherwise.
No borrowed numbers, no estimates.
Results (avg of 3 runs):
wrapper bare cold warm warm vs bare
sbt-direct oneshot 1661ms 4922ms 3600ms SLOWER
sbt-direct bsp 1661ms 1285ms 131ms ~13× faster
dotnet-direct 1172ms 1739ms 555ms ~2× faster
prettier-direct 211ms 1278ms 95ms ~2× faster
eslint-direct 274ms 1252ms 88ms ~3× faster
scalafmt-direct 86ms 1243ms 112ms ~same
Honest interpretation included:
- sbt oneshot is SLOWER than bare sbt (sbt's own launcher daemon
caches classpath; our oneshot spawns fresh JVM each call).
Documented as "use bsp mode whenever possible".
- dotnet-direct is within 2× of bare because MSBuild build-server
persists across bare invocations too. Shipped for uniform CLI
contract + calls.log observability, not speed.
- scalafmt-direct is ~same as bare (native binary is already fast);
ship for consistency, not speed.
- sbt bsp warm is the headline win: 131ms vs bare's 207ms-4.5s
variable range.
- prettier + eslint warm are 2-3× wins because the daemon keeps
require()d packages hot in memory.
sbt bsp "cold" caveat called out explicitly: 1285ms is first /call
after fresh state dir WITH a still-warm sbt JVM from prior session.
Genuine from-scratch cold (Ivy/Coursier resolve + JVM boot + BSP
init) is 15-90s on a fresh checkout. All subsequent calls land in
the warm band.
Bench script lived in /tmp/claude-501/bench.sh; not committed (it's
machine-specific timing infrastructure, not part of the repo's test
surface).1 parent 461a8b2 commit 14873d6
1 file changed
Lines changed: 56 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
78 | 134 | | |
79 | 135 | | |
80 | 136 | | |
| |||
0 commit comments