perf(index): serve Electrum reads from transaction positions instead of scanning blocks

.gitignore

@@ -21,3 +21,6 @@ core.*
 !PLAN.md
 !DEVIATIONS.md
 .antigravitycli/
+
+# lean-ctx local tooling cache
+.lean-ctx/

CONCEPTS.md

@@ -267,5 +267,14 @@ A window sized by whichever of two caps binds first. A count alone is wrong wher
 ### Identity-bearing key
 A key that distinguishes which producer wrote a row, as opposed to one that merely locates it. The index's funding, spending, and txid keys are an 8-byte prefix plus a height, so two blocks at one height that share an output script derive identical keys, and rolling the first back a second time deletes the second's rows. The block-header row is identity-bearing because its key is the 80-byte serialized header and the block hash is the double-SHA256 of exactly those bytes. Checking it before deleting is a proxy for rekeying the other three families, taken because rekeying breaks the electrs-compatible layout and forces a reindex.
 
+### Prefix-row rescan cost
+The cost shape of every lossy-prefix index resolver: read the block once per matching row, deserialize it whole, then hash every output script in it to recover what the 8-byte prefix threw away. Measured, not asserted — `resolve_script_history` rises 63.9x for a 64x rise in funding rows and 3.6x for 4x the block bytes, so the two terms are linear and they multiply. The consequence is that per-row work is bounded by block size rather than by how many transactions actually matched, which is why an address funded at 64 heights costs 86.53 ms end-to-end against a 30 ms budget. `resolve_unspent_outputs` pays roughly double because it computes a txid for every transaction before checking any script, while `resolve_script_history` computes one only after a match. The fix is not a cheaper scan: it is carrying the matching transactions' byte positions in the row value, which is unused today, so resolution reads only those ranges and the exact-check survives untouched. See *Identity-bearing key* for what the same lossy prefix costs on the rollback side.
+
+### All-or-scan position fallback
+The invariant that lets index row values carry transaction byte positions without carrying block identity. Funding and txid keys are an 8-byte prefix plus a height, so a superseded block at the same height leaves rows whose positions point into a different block's body — and because the reader exact-checks what it decodes, the failure is a silently short result rather than an error, which is the worst shape a read path can have. Rather than pay 8 bytes per row for a block tag (measured: 0.66x on top of the 1.67x the positions already cost), the reader **falls back to a full block scan the moment any single position fails to resolve**, and trusts the position list only when every one of them matches. Stale offsets land at arbitrary points in unrelated bytes and essentially never decode to a matching transaction, so they take the fallback; an 8-byte prefix collision between two scripthashes takes it too, correctly, once per 2^64 pairs. The rule that makes this safe is that a failed position is never *skipped* — skipping one and keeping the rest is exactly how a partial result gets reported as a complete one. Accepts one residual: a stale offset landing precisely on a transaction boundary whose transaction also matches, while a different transaction in that block matches too. See *Prefix-row rescan cost* for what the positions buy and *Identity-bearing key* for the same lossy prefix on the write side.
+
+### Paired-arm benchmark
+A Criterion group holding the before and after implementations of one change over one identical fixture, so the ratio comes from a single run. Adopted because a stored baseline cannot be trusted across a rebuild, and because the before implementation is wanted anyway as the equivalence oracle — the same function serves both roles. Its second use is diagnostic: while both arms still call the same code, their spread *is* the harness noise floor, measured rather than assumed. That reading is what disqualified the 64-height `subscribe` and `get_balance` groups, whose identical arms differ by 2.0-2.4x on a laptop and therefore cannot resolve a 1.05x gate, while `get_history` held within 1%. A group that cannot resolve the gate does not get to report a win.
+
 ### Resolution-time sampling
 Recording a statistic when its outcome is known rather than when the subject arrives. The fee estimator counted a transaction against every confirmation target the moment it entered, so a fresh arrival was already a failure at every target and a burst silenced the estimator before anything had missed a deadline. It also broke the decay: the denominator had been decaying since entry while a confirmation arrived undecayed, reporting 81 successes in 100 as roughly 85%. Sampling numerator and denominator together at the moment a target resolves fixes both, because they then decay from the same block. The counterpart rule is that a subject leaving for an unrelated reason is untracked without being sampled: an eviction says something about the mempool, not about whether the transaction would have confirmed.

DEVIATIONS.md

@@ -255,3 +255,53 @@ serving remains deferred.
 - **G14 empirical validation still deferred.** The `faster than Bitcoin
   Core` claim requires multi-day same-window live mainnet IBD against
   `bitcoin-rs` and `bitcoind`. Operator responsibility.
+
+## §8 — Task 8: index rows carry transaction byte positions
+
+`PLAN.md` Task 8 specifies porting electrs verbatim, and electrs writes
+`(8-byte prefix || height)` keys with **empty** values. This implementation puts
+a packed `TxPosition[n]` in that unused value: the `(offset, length)` byte range
+of every transaction that produced the row, within its block's serialized body.
+
+### Why
+
+Resolution was `O(funding rows x block size)`. `Indexer::resolve_script_history`
+loaded and fully decoded the block once per row, then SHA256-hashed every output
+script in it. Measured on synthetic fixtures, the cost rose 63.9x for 64x the
+rows and 3.6x for 4x the block bytes — the two terms are linear and they
+multiply. End-to-end, Electrum `blockchain.scripthash.get_history` cost 86.53 ms
+for an address funded at 64 heights, against a G14 budget of 30 ms.
+
+With positions the resolver reads only the named byte ranges. The block-size term
+disappears: at 8 funding heights the same call costs 8.95 µs over 250 KB blocks
+and 9.12 µs over 1 MB blocks. Full numbers, method, and mutation coverage are in
+[`docs/benchmarks/index-read-path.md`](docs/benchmarks/index-read-path.md).
+
+### What this costs
+
+Funding and `TxConfirmed` row storage goes from 12 bytes per row (key only) to
+20 (12 key + 8 value), a measured **1.67x** on those two families, uncompressed.
+Spending rows are unchanged: nothing resolves them back to transactions today.
+
+### Compatibility
+
+Keys, key ordering and row counts are untouched, so an existing index keeps
+working — a row with an empty value takes the whole-block scan path, which is the
+verbatim electrs behaviour and is retained as `*_scan`. Nothing forces a reindex;
+clearing the index directory and re-syncing is what earns the fast path.
+
+`ColumnFamily::UtxoMeta` carries an `index:format_version` marker, adopted only
+when the index is empty. A populated index without one is reported as
+`IndexFormat::Legacy` and the node logs a startup warning naming the directory to
+delete. It does not refuse to start: reads stay correct either way.
+
+### The rule that makes it safe
+
+Funding and txid keys carry no block identity, so a superseded block at the same
+height leaves rows pointing into a different block's bytes. Rather than pay 8
+more bytes per row for a block tag, the reader **falls back to a full block scan
+the moment any single position fails to resolve**, and never skips a failed
+position while keeping the rest. See the *All-or-scan position fallback* concept
+in `CONCEPTS.md`. The residual accepted: a stale offset landing exactly on a
+transaction boundary whose transaction also matches, while a different
+transaction in that block matches too.

crates/electrum/Cargo.toml

@@ -39,4 +39,10 @@ tracing.workspace = true
 [dev-dependencies]
 proptest.workspace = true
 serde_json.workspace = true
+criterion.workspace = true
 tempfile = "3"
+
+[[bench]]
+name = "electrum_methods"
+harness = false
+required-features = ["rocksdb"]