Skip to content

Commit 8ddb0e6

Browse files
committed
docs(perf): resolve the contradictory headline, record the closing program
The measurement table still told readers to quote 121.9s vs 59.6s = 2.05x while the harness section below said 1.42x and marked 2.05x superseded. Fixed: the table now carries the local-block-file row and points at the harness section. Also records what the harness fix revealed. Apply is 76.7s against Core's 55.80s, a 20.9s gap, and the identified per-stage deltas now sum to 20.6s -- the gap is fully accounted for, with no large unexplained remainder. Closing all of them reaches 64.0s = 1.07x Core. Every item is individually 1.04-1.07x, at or under the single-candidate gate, so the next session should run them as one program gated on the cumulative number rather than as five candidates, start with merkle (pure function, immediately falsified), and settle block_body_persist on policy rather than deleting it to win a benchmark.
1 parent e877f28 commit 8ddb0e6

1 file changed

Lines changed: 25 additions & 2 deletions

File tree

docs/solutions/performance-issues/txid-parallelization-delivers-2x-but-core-still-leads.md

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,10 @@ Same machine (128 cores), serial runs, local REST blocks (fjall, full verificati
4444
| Core 31.0 || ~~67s~~ **superseded** | 2240 | n/a | 2026-06-09 debug.log, unknown load | do not quote; re-derived below |
4545
| Core 31.0 || **59.6s median** (59.6, 59.4, 60.2) | 2517 | n/a | `taskset -c 0-31` 3× interleaved with rs, same idle host | `-reindex-chainstate -assumevalid=0 -connect=0 -stopatheight=150000` |
4646
| bitcoin-rs | matched pair | 132.2s median (132.2, 134.5, 131.6) | 1135 | 226 MB | same interleaved series | apply 95.3s; the rest is REST fetch |
47-
| bitcoin-rs | one-shot kernel block parse | **121.9s median** (121.9, 124.4, 120.6) | **1231** | 226 MB | `taskset -c 0-31` 3× paired against the prior binary | apply 82.0s, `script_prepare` 4.29s |
47+
| bitcoin-rs | one-shot kernel block parse, REST source | 121.9s median (121.9, 124.4, 120.6) | 1231 | 226 MB | `taskset -c 0-31` 3× paired against the prior binary | apply 82.0s, `script_prepare` 4.29s |
48+
| bitcoin-rs | **local block file source** | **84.6s median** (84.2, 84.6, 86.5) | **1774** | **224 MB** | `taskset -c 0-31`| apply 76.7s; block source matched to Core |
4849

49-
**Quote the matched pair, not a cross-run ratio.** After the one-shot parse: **121.9s vs Core 59.6s = 2.05×** (apply 82.0s alone is 1.38× Core's whole run). Total self-improvement over the `4700c25` baseline is **3.4×**.
50+
**Quote 84.6s vs 59.6s = 1.42×.** Every row above it fetched blocks over REST while Core read local `blk*.dat` files, so those ratios measure the harness as much as the engine — see the harness section below. Total self-improvement over the `4700c25` baseline is **4.6×**.
5051

5152
### GoCoin: bitcoin-rs wins, and by more than the raw numbers show
5253

@@ -334,6 +335,28 @@ Why ours is ~1.5× slower on the same code is unexplained and worth knowing, but
334335

335336
Do not re-open script verification: it is a measured tie, and four marshalling micro-optimizations plus a pool-width and threshold sweep are already closed above.
336337

338+
## The gap is now fully accounted for, and it is a program of small items
339+
340+
With the harness matched, the arithmetic closes for the first time. Apply is 76.7s against Core's 55.80s, a **20.9s** gap, and the identified per-stage deltas sum to **20.6s** — there is no longer a large unexplained remainder hiding in the measurement.
341+
342+
| Stage | bitcoin-rs | Core | delta | alone |
343+
|---|---|---|---|---|
344+
| script verification | 36.47s | 36.07s | ~0 | tie |
345+
| `script_prepare` + `script_resolution` | 5.80s | folded into Connect | 5.80s | 1.074× |
346+
| `block_body_persist` | 4.18s | none in reindex | 4.18s | 1.052× |
347+
| block parse | 10.88s | 7.18s | 3.70s | 1.046× |
348+
| `utxo_commit` | 6.10s | 2.59s | 3.51s | 1.043× |
349+
| `block_rules` / merkle | 4.79s | 1.41s | 3.38s | 1.042× |
350+
| **all together** | | | **20.6s** | **1.32×** |
351+
352+
Closing all of them lands at **64.0s against Core's 59.6s = 1.07×**, which is parity within a rounding of the noise band.
353+
354+
**This changes how the remaining work should be run.** Every item is individually 1.04–1.07×, at or under the 1.05× single-candidate gate, so none of them will ever look convincing on its own — and at ±5% single-run noise on an 84.6s run, a 3.5s effect is at the edge of what a 3× median can resolve. The next session should therefore:
355+
356+
1. Treat these as **one program, not five candidates**. Gate the program on the cumulative number, and use paired interleaved runs with more than three repetitions to resolve each step.
357+
2. Start with `block_rules` (merkle root over scalar SHA-256 against Core's AVX2) — it is a pure function, immediately falsified by any error since every block checks it, and therefore the lowest-risk of the five.
358+
3. Decide `block_body_persist` on policy, not performance: Core's reindex writes no blocks, so it is either excluded from the ratio or matched by having Core do it. Do not simply delete it to win a benchmark; a real node must store blocks.
359+
337360
## Guidance
338361

339362
1. **Attribute a stage by disabling it, not by reading a profiler.** `perf` was unavailable here (`perf_event_paranoid=4`, no sudo), but the open question — is `script_parallel`'s ~0.93 ms/block genuine secp256k1 work or rayon dispatch overhead? — is binary, so forcing `MIN_PARALLEL_SCRIPT_CHECKS = usize::MAX` answered it in one run: the replay went 173.1s → **313.3s** and the stage 63s → **227.6s**. Genuine crypto. That immediately reframed the number: 227.6s → 63s is only **3.6× from a 16-thread pool**, so the pool width was the binding constraint, and widening it to 32 bought 1.10× (`0e2dda5`). Prefer this disable-the-stage technique whenever a hypothesis is binary; it needs no tooling and cannot be argued with.

0 commit comments

Comments
 (0)