feat(network)!: fully migrate replay transport to p2p network by itegulov · Pull Request #873 · matter-labs/zksync-os-server

itegulov · 2026-02-11T09:46:31Z

Summary

This PR gets rid of legacy block replay transport protocol in favour of the new devp2p-driven implementation. Most of the implementation work was done some time ago before this PR, here I am mostly ironing out remaining issues and finalizing the migration.

I tested this manually on the new bootstrap node with static IP address. Hit me in DMs to give you an instruction on how to do it (don't want to expose the IP address here before we finalize everything).

Breaking Changes

Removed sequencer_block_replay_download_address
Removed sequencer_block_replay_server_enabled
Removed sequencer_block_replay_server_address
general_node_role is a new config parameter that is required to be set on ENs
Block replay transport is no longer hosted over HTTP on port 3053, instead devp2p protocol is used for p2p communication over TCP (port 3060 by default)

Rollout Instructions

Delete sequencer_block_replay_* config parameters from all nodes
If your environment does not use external nodes (replay transport was unused before) - no further actions are required, otherwise continue down the list
Make sure your cluster has at least one node with static IP address (it can be local as long as you do not want to connect external nodes from outside)
Make sure all external nodes have general_node_role set to external
Make sure networking is enabled and configured properly for all nodes (main and external), example:

network:
  # Enables P2P networking, required for syncing ENs
  enabled: "true"
  # Secret key used for node identity in the network, can be randomly generated locally.
  # MUST be different for different nodes
  secret_key: "21b0ee131240821c39627c39d0fdde5edbda968c5877f5b63c5c542f267b5349"
  # Set to node's static IP address if there is one, leave unconfigured otherwise (defaults to 0.0.0.0)
  address: "14.28.133.107"
  # Comma-separated list of node records that have static IP addresses.
  # Can be left unconfigured if this is the node with static IP address. Otherwise this MUST contain at least one entry.
  # Format is `enode://<node ID>@<IP address>:<port>` where `node ID` is public key corresponding to that node's secret key (see above).
  # NOTE that node with static IP address does not necessarily need to be main node
  boot_nodes: "enode://dbd18888f17bad7df7fa958b57f4993f47312ba5364508fd0d9027e62ea17a037ca6985d6b0969c4341f1d4f8763a802785961989d07b1fb5373ced9d43969f6@14.28.133.115:3060"

github-actions · 2026-02-11T09:56:49Z

Test results

187 tests 187 ✅ 18m 21s ⏱️
27 suites 0 💤
1 files 0 ❌

Results for commit e12767b.

♻️ This comment has been updated with latest results.

node/bin/src/lib.rs

RomanBrodetski

Looks good!
Would be great if @popzxc could also review as he was following this whole initiative 🙏

node/bin/src/config/mod.rs

popzxc

Overall, great job! Left some nits, but don't think that they're worth blocking the merge; will be fine with merging as-is.

docs/src/setup/external_node.md

lib/network/src/protocol.rs

docs/src/setup/external_node.md

node/bin/src/lib.rs

node/bin/src/main.rs

node/bin/src/replay_transport.rs

## Summary Follow-up to #873 with a few improvements: * Bounded channel for network->sequencer communication to ensure network does not save insane amount of records before sequencer is up (which can take a couple of minutes depending on L1 state recovery). * Shared `starting_block` state for all connections. This makes sure we do not restart streaming from the same block again and again on reconnection to main node. * Tuned ban duration and backoff duration to better match our current needs * Removed unused replay wire formats (forgot to do it in last PR) This is the version currently deployed to stage which has been running fine for the last ~24h

🤖 I have created a release *beep* *boop* --- ## [0.16.0](v0.15.1...v0.16.0) (2026-02-25) ### ⚠ BREAKING CHANGES * **network:** fully migrate replay transport to p2p network ([#873](#873)) * change api l2 l1 log format ([#875](#875)) ### Features * add block hash to revm divergence panic message ([#880](#880)) ([92a9eaf](92a9eaf)) * **batch-verification:** make HTTPS connection a 2-way stream ([#862](#862)) ([a96e9a0](a96e9a0)) * change api l2 l1 log format ([#875](#875)) ([26ea56f](26ea56f)) * index reverted blocks by hash ([#867](#867)) ([8e360fb](8e360fb)) * **mempool:** rewrite via in-memory subpools ([#869](#869)) ([b3bbca8](b3bbca8)) * **network:** bounded channel + shared starting block state ([#884](#884)) ([5de34e2](5de34e2)) * **network:** fully migrate replay transport to p2p network ([#873](#873)) ([a8e963a](a8e963a)) ### Bug Fixes * Apply fixes for cargo deny ([#892](#892)) ([e4eef3c](e4eef3c)) * Commit after each tx in revm consistency checker ([#898](#898)) ([384ff31](384ff31)) * get rid of broadcast in mempool ([#910](#910)) ([01b53fd](01b53fd)) * remove transaction r and s paddings ([#890](#890)) ([3079e59](3079e59)) * **rpc:** return hex-encoded subscription ids ([#877](#877)) ([0dbc703](0dbc703)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: zksync-era-bot <zksync-era-bot@users.noreply.github.com>

itegulov added 9 commits February 11, 2026 16:06

feat(network): untie defaults from local setup

38385c6

feat!: introduce general_node_role config for MN/EN switch

88caa26

feat!: migrate from legacy replay transport to p2p transport

cab4b24

feat(network): continuously stream replay records

902a3a9

test: adapt integration tests

e7aa2c8

feat(network): only stream records for EN<->MN pairs

fb09519

feat(network): support record overrides

69bf377

feat(network): respect starting_block determined during recovery

a54343b

chore: update docs

dbdfa41

itegulov requested review from Artemka374, RomanBrodetski, perekopskiy and slowli February 11, 2026 09:46

RomanBrodetski reviewed Feb 11, 2026

View reviewed changes

node/bin/src/lib.rs Outdated Show resolved Hide resolved

RomanBrodetski previously approved these changes Feb 11, 2026

View reviewed changes

RomanBrodetski reviewed Feb 11, 2026

View reviewed changes

node/bin/src/config/mod.rs Show resolved Hide resolved

itegulov added 2 commits February 12, 2026 17:29

feat(network): update local ENR more aggressively

2f79b22

Merge branch 'main' into di/p2p-block-replay

1cfc323

itegulov dismissed RomanBrodetski’s stale review via 1cfc323 February 13, 2026 05:44

chore: add a link to PR with rollout instructions

672cd29

popzxc previously approved these changes Feb 13, 2026

View reviewed changes

itegulov added 3 commits February 13, 2026 19:06

chore: serialize NodeRole as main/external

60fe8ef

chore: impl Display for NodeRole

5e5f4b5

chore: remove irrelevant zkstack config logic

9e06914

itegulov dismissed popzxc’s stale review via 9e06914 February 13, 2026 08:11

itegulov enabled auto-merge (squash) February 13, 2026 08:13

popzxc previously approved these changes Feb 13, 2026

View reviewed changes

chore: clippy

e12767b

itegulov dismissed popzxc’s stale review via e12767b February 13, 2026 09:52

popzxc approved these changes Feb 13, 2026

View reviewed changes

itegulov disabled auto-merge February 13, 2026 10:00

itegulov merged commit a8e963a into main Feb 17, 2026
17 checks passed

itegulov deleted the di/p2p-block-replay branch February 17, 2026 02:45

zksync-era-bot mentioned this pull request Feb 12, 2026

chore(main): release 0.16.0 #878

Merged

itegulov mentioned this pull request Feb 17, 2026

feat(network): bounded channel + shared starting block state #884

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(network)!: fully migrate replay transport to p2p network#873

feat(network)!: fully migrate replay transport to p2p network#873
itegulov merged 16 commits intomainfrom
di/p2p-block-replay

itegulov commented Feb 11, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

RomanBrodetski left a comment

Uh oh!

Uh oh!

popzxc left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

itegulov commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking Changes

Rollout Instructions

Uh oh!

github-actions bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test results

Uh oh!

Uh oh!

RomanBrodetski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

popzxc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

itegulov commented Feb 11, 2026 •

edited

Loading

github-actions bot commented Feb 11, 2026 •

edited

Loading