Skip to content

fix(bootnodoor): hand discv5-only EL clients the EL ENR instead of the enode - #1462

Merged
barnabasbusa merged 4 commits into
bbusa/el-discv5-only-enr-bootnodesfrom
bbusa/el-discv5-bootnodoor-el-enr
Jul 31, 2026
Merged

fix(bootnodoor): hand discv5-only EL clients the EL ENR instead of the enode#1462
barnabasbusa merged 4 commits into
bbusa/el-discv5-only-enr-bootnodesfrom
bbusa/el-discv5-bootnodoor-el-enr

Conversation

@pk910

@pk910 pk910 commented Jul 30, 2026

Copy link
Copy Markdown
Member

Targets #1461. Merges #1451 and closes the gap between the two: with ELs defaulting to discv5-only, the bootnodoor override still handed every EL client an enode:// URL, which feeds a disabled protocol — and besu in V5 mode rejects non-ENR bootnodes outright, so a discv5-only enclave with bootnodoor enabled had no working EL bootstrap at all.

What

  • launch_bootnodoor also fetches /el-enr and returns (cl_enr, enode, el_enr). /el-enr is the EL identity's record with the CL-only eth2 field stripped (mirroring why feat: update bootnodoor integration for v0.0.3 #1451 switched the CL side to /cl-enr), and stays correct with separate_keys: true, where the CL ENR is a different node identity altogether.
  • el_launcher.launch selects the override per client: the seven discv5-only launchers (geth, besu, erigon, nethermind, reth, nimbus-eth1, ethrex) get the EL ENR — their bootnodoor_enode parameter is renamed to bootnodoor_el_enr — while discv4-only ethereumjs keeps the enode (bootnodoor still answers discv4 on the EL identity, so that pairing keeps working).

Testing

kurtosis lint clean. Two live enclaves on this branch, all images freshly pulled:

geth + besu + bootnodoor — both ELs received --bootnodes=enr:… matching bootnodoor's /el-enr, peered within 10s (besu is the strict case: it hard-rejects enode bootnodes in V5 mode), both imported blocks.

7-EL matrix + bootnodoor (geth, besu, erigon, reth, nethermind, nimbus-eth1, ethrex — bootnodoor as sole bootnode, no static peer bootnodes):

el-1-geth        peers=5
el-2-besu        peers=5
el-3-erigon      peers=0   <- see below
el-4-reth        peers=5
el-5-nethermind  peers=5
el-6-nimbus      peers=5
el-7-ethrex      peers=5

Six clients registered in bootnodoor's EL table and formed a full mesh through it (peers=5 = everyone except erigon); all seven lighthouses registered in the CL table and peered. Chain progressed on all nodes.

⚠️ erigon's discv5 is non-functional

Erigon parses the ENR bootnode, logs Setup P2P discovery v4=false v5=true, and binds UDP 30303 — but never sends a single discv5 packet: after 15+ min it is absent even from bootnodoor's inactive-node DB, and it logs [p2p] No GoodPeers forever. #1461's own 7-EL test masked this: erigon showed peers=1 there only because other clients had erigon's ENR in their static bootnode lists and dialed it inbound. Behind bootnodoor — where discovery is the only way to be found — erigon is fully isolated (it still follows the chain via its CL / engine API). Needs an upstream erigon fix; nothing this package can do.

Stale-image traps (relevant to CI on both parent PRs)

  • A cached ethpandaops/bootnodoor:latest still on v0.0.2 404s on /cl-enr, hard-failing feat: update bootnodoor integration for v0.0.3 #1451's fetch.
  • A cached ethpandaops/besu:main predating besu#10800 doesn't know --discovery-mode.
  • ethpandaops/ethereumjs:master (2026-05-19) crashes at startup with crypto.getRandomValues must be defined — unrelated to these PRs (dies generating its node key), but it means the ethereumjs+bootnodoor enode path can't be live-tested until that image is rebuilt on a newer Node.

Devnet EL seeding

#1451 seeded bootnodoor's devnet EL side from enodes.txt only. --el-bootnodes accepts ENRs and enode URLs alike, so the new get_devnet_el_bootnodes merges el_enrs.txt (discv5) with enodes.txt (discv4) into one list and tolerates either file being absent — el_enrs.txt only appears once ethpandaops/ansible-collection-general#568 lands and devnet metadata is regenerated, and discv4-only stragglers stay reachable through the enodes. The merge snippet was verified for all four file-presence combinations (exit 0, clean comma list); the devnet path itself has no live devnet with el_enrs.txt to test against yet.

MysticRyuujin and others added 4 commits July 16, 2026 18:10
- pin --enr-ip (nat_exit_ip, or K8S_POD_IP on kubernetes) so the ENR no longer rewrites to the host WAN address learned from PONGs
- pass --cl-bootnodes/--el-bootnodes (v0.0.3 split the removed --bootnodes flag) and feed enodes.txt to the EL side
- fetch /cl-enr for the CL bootnode override (strips the EL-only eth field)
- publish HTTP/discovery ports via port_publisher.additional_services, binding the published port number so the ENR is valid in-enclave and externally
- scrape /metrics via prometheus additional_metrics_jobs
- optional bootnodoor_params.separate_keys for distinct EL/CL identities
…e enode

With ELs defaulting to discv5-only, the enode:// bootnode override from
bootnodoor feeds a disabled protocol (and besu rejects non-ENR bootnodes
outright in V5 mode). Fetch /el-enr — the EL identity's record with the
CL-only eth2 field stripped, which is also correct with separate_keys —
and use it for every EL client except discv4-only ethereumjs, which keeps
the enode.
--el-bootnodes accepts ENRs and enode URLs alike, so merge el_enrs.txt
(discv5) with enodes.txt (discv4) and tolerate either file being absent:
el_enrs.txt only appears once ansible-collection-general#568 lands, and
discv4-only stragglers stay reachable through the enodes.
@pk910
pk910 marked this pull request as ready for review July 31, 2026 01:59
@barnabasbusa
barnabasbusa merged commit 234b662 into bbusa/el-discv5-only-enr-bootnodes Jul 31, 2026
17 of 19 checks passed
@barnabasbusa
barnabasbusa deleted the bbusa/el-discv5-bootnodoor-el-enr branch July 31, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants