Skip to content

Commit a698e2c

Browse files
pragmaximclaude
pragmaxim
and
claude
committed
docs(eth): map the two EVM pending-transaction stores
docs/evm-send.md draws only the private path, but its removal funnel clears the wrapped Blockbook mempool too, and that store is never shown - so neither its own ingest (newPendingTransactions feed, resync snapshot) nor its own exits (block connect, timeout sweep, backend-missing removal) are visible, and nor is the fact that a private tx lives in both stores at once. Three diagrams: the broadcast/ingest/eviction lifecycle across both stores, the per-entry reconcile ladder, and the serve path that combines them. Prose is limited to the store comparison table and the two load-bearing couplings - a private tx must be cached before the mempool add can index it, and the cache must expire before the mempool. The reconcile decisions are a separate diagram rather than a subgraph in the first: nodes with no edges between them share a dagre rank and lay out in one very wide row, which a subgraph does not constrain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 9bbfb3b commit a698e2c

2 files changed

Lines changed: 201 additions & 0 deletions

File tree

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,5 @@
1111
* [API (Tron specifics)](/docs/api-tron.md) – Tron-specific behavior and data extensions for API V2
1212
* [Sync](/docs/sync.md) – Sync-loop architecture and the `missingBlockRetry` troubleshooting knobs
1313
* [EVM send](/docs/evm-send.md) – EVM transaction broadcast through the private send-tx relay and its pending-transaction cache
14+
* [EVM pending-transaction stores](/docs/evm-send-mempools.md) – Map of the two EVM pending stores: the Blockbook mempool and the private/MEV cache
1415
* [Testing](/docs/testing.md) – Description of tests used during Blockbook development

‎docs/evm-send-mempools.md‎

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# EVM pending-transaction stores
2+
3+
Blockbook keeps EVM pending transactions in **two** stores when a coin is configured with a private
4+
send-tx relay (`*_ALTERNATIVE_SENDTX_URLS`, `*_ALTERNATIVE_SENDTX_ONLY`,
5+
`*_ALTERNATIVE_FETCH_MEMPOOL_TX`). Without a relay only the Blockbook mempool exists and the private
6+
path below is skipped entirely. The broadcast flow and its invariants are described in
7+
[evm-send.md](/docs/evm-send.md); this page is the map of the two stores.
8+
9+
| | Blockbook mempool | MEV / private cache |
10+
|---|---|---|
11+
| Type | `bchain.MempoolEthereumType` (`b.Mempool`) | `eth.AlternativeSendTxProvider.mempoolTxs` |
12+
| Holds | txid + address index (+ token transfers) | full `RpcTransaction` body, sender, nonce, send generation |
13+
| Populated from | `newPendingTransactions` WS feed; the resync snapshot; own sends when `disableMempoolSync` | only sends a relay ACKed, in `ALTERNATIVE_SENDTX_ONLY` + `ALTERNATIVE_FETCH_MEMPOOL_TX` mode |
14+
| Serves | address and xpub txs (`GetAddrDescTransactions`), wallet `NewTx` pushes | tx bodies on `GetTransaction`, the pending-nonce floor |
15+
| Retention | `mempoolTxTimeout` — 10 min when a relay is configured, else `mempoolTxTimeoutHours` | `alternativeMempoolTxTimeout` — 5 min |
16+
| Reconciled | `Resync` every ~60 s; timeout sweep at most every 10 min | `reconcileMempoolTxs` every 1 min, against the relay |
17+
18+
A private transaction is in **both**: the cache holds the body and is the source of truth, the
19+
wrapped mempool holds the index built from it. A public transaction is only ever in the mempool.
20+
21+
Two couplings between the stores are load-bearing:
22+
23+
- `AddTransactionToMempool` builds its address index through `GetTransactionForMempool` →
24+
`GetTransaction`, which reads the cache first. A private transaction must therefore be cached
25+
*before* it is added to the wrapped mempool, or it cannot be indexed at all.
26+
- The cache must expire **before** the wrapped mempool. Every cache exit clears the wrapped mempool
27+
too, but the mempool's own timeout sweep does not clear the cache; inverted, a private transaction
28+
loses its address index while still being served as pending. The defaults are ordered correctly
29+
and `CreateMempool` warns when an explicit configuration inverts them.
30+
31+
## Broadcast, ingest and eviction
32+
33+
```mermaid
34+
%%{init: {"theme": "base", "themeVariables": {"lineColor": "#6b7280", "primaryTextColor": "#111827"}}}%%
35+
flowchart TD
36+
send["SendRawTransaction(hex, disableAlternativeRPC)"]
37+
route{"relay configured<br/>and not disabled?"}
38+
relay["broadcast to every relay URL<br/>eth_sendRawTransaction"]
39+
acc{"any relay URL accepted?"}
40+
only{"ALTERNATIVE_SENDTX_ONLY?"}
41+
fail["return relay error<br/>no cache path, no fetch-back"]
42+
primary["primary backend<br/>eth_sendRawTransaction"]
43+
reg["registerSuccessfulSend<br/>sender + accepting URL + nonce slot<br/>assign send generation"]
44+
ackevict["evictReplacedByNonce<br/>retire same from+nonce predecessor<br/>on ACK, generation-ordered"]
45+
handle["handleMempoolTransaction<br/>fetch-back eth_getTransactionByHash<br/>skipped if a newer send holds the slot"]
46+
47+
ws["eth_subscribe newPendingTransactions<br/>skipped when disableMempoolSync"]
48+
snap["startup and Resync snapshot<br/>eth_getBlockByNumber pending<br/>only when queryBackendOnMempoolResync"]
49+
50+
alt[("MEV / private cache<br/>full tx bodies<br/>timeout 5 min")]
51+
pub[("Blockbook mempool<br/>txids + address index<br/>timeout 10 min with relay")]
52+
53+
altrec["reconcileMempoolTxs, every 1 min<br/>evicts mined, nonce_superseded, timeout<br/>keeps provider_missing until timeout"]
54+
pubrec["Mempool Resync, every 60 s<br/>timeout sweep at most every 10 min<br/>plus backend-missing removal"]
55+
56+
readalt["GetTransaction read path<br/>entry past the cache timeout"]
57+
blk["GetBlock: tx in a connected block"]
58+
readmined["GetTransaction: mined or unknown"]
59+
60+
altrm[("removeMempoolTx<br/>cache delete decides the race<br/>release nonce routing, metered once")]
61+
bothrm[("removeTransactionFromMempool<br/>clears BOTH stores")]
62+
63+
send --> route
64+
route -- "no" --> primary
65+
route -- "yes" --> relay --> acc
66+
acc -- "no" --> only
67+
only -- "yes" --> fail
68+
only -- "no" --> primary
69+
acc -- "yes" --> reg --> ackevict --> handle
70+
ackevict -. "predecessor" .-> altrm
71+
handle -- "1. cache the body" --> alt
72+
handle -- "2. AddTransactionToMempool,<br/>index built by reading the cache,<br/>then push NewTx" --> pub
73+
primary -. "only when disableMempoolSync" .-> pub
74+
ws --> pub
75+
snap --> pub
76+
77+
alt --> altrec --> altrm
78+
readalt --> altrm
79+
altrm -- "delegate, already metered" --> bothrm
80+
blk -- "metered sync_removed" --> bothrm
81+
readmined -- "metered sync_removed" --> bothrm
82+
bothrm -- "delete" --> alt
83+
bothrm -- "delete" --> pub
84+
pub --> pubrec -- "wrapped mempool only" --> pub
85+
86+
classDef step fill:#e7f0ff,stroke:#4078c0,color:#10243e;
87+
classDef mev fill:#f3e8ff,stroke:#7c3aed,color:#2b1148;
88+
classDef pubstore fill:#e8f7ed,stroke:#2e8b57,color:#0b2c19;
89+
classDef sink fill:#fff7e6,stroke:#b8860b,color:#3a2a00;
90+
classDef error fill:#ffecec,stroke:#c03535,color:#3b0a0a;
91+
class send,route,relay,acc,only,primary,reg,ackevict,handle,ws,snap,altrec,pubrec,readalt,blk,readmined step;
92+
class alt mev;
93+
class pub pubstore;
94+
class altrm,bothrm sink;
95+
class fail error;
96+
```
97+
98+
## What the cache reconcile decides, per entry
99+
100+
Evaluated in this order once a minute for every cached transaction. The labels are the `action`
101+
values of `blockbook_eth_alternative_mempool_reconciliation_events_total`.
102+
103+
```mermaid
104+
%%{init: {"theme": "base", "themeVariables": {"lineColor": "#6b7280", "primaryTextColor": "#111827"}}}%%
105+
flowchart TD
106+
e["cached entry, 1 min tick"]
107+
f{"age under 1 min?"}
108+
keepFresh["keep: skipped_fresh"]
109+
q["relay eth_getTransactionByHash"]
110+
qerr{"past 5 min timeout?"}
111+
evTo["evict: timeout"]
112+
keepErr["keep: provider_error"]
113+
m{"blockNumber set?"}
114+
evMined["evict: mined"]
115+
s{"confirmed nonce above tx nonce?<br/>relay eth_getTransactionCount latest"}
116+
evSup["evict: nonce_superseded"]
117+
k{"still surfaced by the relay?"}
118+
kto{"past 5 min timeout?"}
119+
evMiss["evict: provider_missing"]
120+
keepMiss["keep: provider_missing_pending<br/>an empty probe is not authoritative"]
121+
yto{"past 5 min timeout?"}
122+
evTo2["evict: timeout"]
123+
keepK["keep: kept"]
124+
125+
e --> f
126+
f -- "yes" --> keepFresh
127+
f -- "no" --> q
128+
q -- "error" --> qerr
129+
qerr -- "yes" --> evTo
130+
qerr -- "no" --> keepErr
131+
q -- "answered" --> m
132+
m -- "yes" --> evMined
133+
m -- "no" --> s
134+
s -- "yes" --> evSup
135+
s -- "no" --> k
136+
k -- "no" --> kto
137+
kto -- "yes" --> evMiss
138+
kto -- "no" --> keepMiss
139+
k -- "yes" --> yto
140+
yto -- "yes" --> evTo2
141+
yto -- "no" --> keepK
142+
143+
classDef step fill:#e7f0ff,stroke:#4078c0,color:#10243e;
144+
classDef keep fill:#e8f7ed,stroke:#2e8b57,color:#0b2c19;
145+
classDef evict fill:#fff7e6,stroke:#b8860b,color:#3a2a00;
146+
class e,f,q,qerr,m,s,k,kto,yto step;
147+
class keepFresh,keepErr,keepMiss,keepK keep;
148+
class evTo,evTo2,evMined,evSup,evMiss evict;
149+
```
150+
151+
Every `evict:` box funnels through `removeMempoolTx` in the first diagram. In practice a mined
152+
private transaction is usually cleared by block sync first, counted as `sync_removed`, before the
153+
next probe reaches the `mined` branch here.
154+
155+
## How the two stores are read back
156+
157+
Only addresses that sent through the relay within the cache retention are routed to it
158+
(`useForNonces`); everything else is served by the primary backend.
159+
160+
```mermaid
161+
%%{init: {"theme": "base", "themeVariables": {"lineColor": "#6b7280", "primaryTextColor": "#111827"}}}%%
162+
flowchart LR
163+
tx["GetTransaction(txid)"]
164+
hit{"in MEV cache<br/>and not expired?"}
165+
body["serve the cached body<br/>relay never asked"]
166+
rpcget["primary eth_getTransactionByHash<br/>then pruned-index recovery"]
167+
168+
addr["address / xpub request"]
169+
idx["Blockbook mempool address index"]
170+
171+
non["EthereumTypeGetNonces(addr)"]
172+
gate{"useForNonces(addr)?<br/>private send within 5 min"}
173+
nrelay["relay eth_getTransactionCount<br/>single accepting URL, batched pending+latest"]
174+
nprim["primary eth_getTransactionCount"]
175+
floor["raiseToPendingFloor<br/>never below highest cached nonce + 1"]
176+
177+
est["EthereumTypeEstimateGas"]
178+
egate{"from set and useForNonces?"}
179+
erelay["relay eth_estimateGas"]
180+
eprim["primary eth_estimateGas"]
181+
182+
tx --> hit
183+
hit -- "yes" --> body
184+
hit -- "no" --> rpcget
185+
addr --> idx -- "per txid" --> tx
186+
non --> gate
187+
gate -- "yes" --> nrelay --> floor
188+
gate -- "no" --> nprim --> floor
189+
est --> egate
190+
egate -- "yes" --> erelay
191+
erelay -. "error or bad result" .-> eprim
192+
egate -- "no" --> eprim
193+
194+
classDef step fill:#e7f0ff,stroke:#4078c0,color:#10243e;
195+
classDef mev fill:#f3e8ff,stroke:#7c3aed,color:#2b1148;
196+
classDef pubstore fill:#e8f7ed,stroke:#2e8b57,color:#0b2c19;
197+
class tx,hit,rpcget,addr,non,gate,nprim,est,egate,erelay,eprim,floor step;
198+
class body,nrelay mev;
199+
class idx pubstore;
200+
```

0 commit comments

Comments
 (0)