From f74d5b09138ae6a77ff07e2062c135a14c86bad2 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Sun, 2 Aug 2026 23:26:24 +0100 Subject: [PATCH 1/8] project proposal --- ...ighthouse-decentralized-checkpoint-sync.md | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 projects/lighthouse-decentralized-checkpoint-sync.md diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md new file mode 100644 index 0000000..d922aec --- /dev/null +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -0,0 +1,192 @@ +# Lighthouse Decentralized Checkpoint Sync + +Trust-minimized checkpoint sync for Ethereum full nodes via light client bootstrap and verifiable state backfill. + +## Motivation + +Checkpoint sync today requires nodes to trust a checkpoint provider — a URL or other source that supplies the state needed to bootstrap. That trust assumption is at odds with a chain that claims to be decentralized. + +This project implements a trust-minimized checkpoint sync pathway. A new node bootstraps as a light client using a hardcoded, network-verified block root (e.g., the first Altair block root), syncs forward to the present using the existing light client protocol, obtains cryptographic proof of a recent agreed-upon checkpoint state, and then backfills that state from the p2p network in verifiable chunks. + +This work primarily affects the sync protocol of full nodes and extends the responsibilities of light client data providers in the network, who will now be required to store and serve certain data they were not required to previously. + +## Project description + +Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, inspired by the light client (LC) sync protocol and guided by ongoing work on light client data backfill ([EIP-7658](https://eips.ethereum.org/EIPS/eip-7658)). The flow for a joining node is: + +1. **Light Client Bootstrap:** The node starts with a trusted block root baked into the client (e.g., the first Altair block root). It requests a `LightClientBootstrap` and initializes a `LightClientStore`. +2. **Forward Sync:** The node requests `LightClientUpdates` by range, syncing forward from the trusted root to the present. It now has a verified recent `beacon_block_root`. +3. **Checkpoint Discovery:** The node requests a `LightClientBeaconSnapshot`, a structure containing a recent, agreed-upon state root and a Merkle proof connecting it to the verified block header. +4. **State Backfill:** The node fetches the `BeaconState` at that state root in fixed-size chunks from multiple peers. Each chunk comes with a Merkle proof verifying its inclusion in the state tree. The node verifies each chunk independently and reassembles the full state. +5. **Full Node Activation:** Once the state is fully fetched and verified, the node has everything it needs to transition to full node duties. + +The core of this project, however, is not just the high-level flow — it is the Lighthouse client infrastructure required to make this flow possible. Our research into the Lighthouse codebase revealed that the storage schema and serving endpoints for light client data already exist (`DBColumn::LightClientUpdate`, `SyncCommitteeBranch`, `SyncCommittee`, and the p2p `LightClientUpdatesByRange` endpoint). The critical gap is that historical data is not persisted due to the following barriers: + +**Recency guard in `import_block_update_metrics_and_events`:** + +```rust +// Do not write to the cache for blocks older than 2 epochs, this helps reduce writes to +// the cache during sync. +if block_delay_total < self.slot_clock.slot_duration() * 64 { + // Store the timestamp of the block being imported into the cache. + self.block_times_cache.write().set_time_imported( + block_root, + current_slot, + block_time_imported, + ); +} +``` + +This is an optimization. When Lighthouse is syncing old blocks, it skips notifying the light client server to avoid extra computation. The assumption was: "nobody needs light client data for old blocks." That assumption now stands as a barrier to what this project aims to achieve. + +**Storage gap in `get_light_client_bootstrap`:** + +```rust +// we currently have no backfill mechanism for these values. +// Therefore, sync_committee_branch and sync_committee are only persisted while a node is synced. +#[allow(clippy::type_complexity)] +pub fn get_light_client_bootstrap( + &self, + store: &BeaconStore, + block_root: &Hash256, + finalized_period: u64, + chain_spec: &ChainSpec, +) -> Result, ForkName)>, BeaconChainError> {...} +``` + +Even if you synced old blocks, Lighthouse doesn't retroactively compute and store the `sync_committee_branch`. It only stores this for the blocks you processed while already synced. + +The combined effect: a Lighthouse node that synced from genesis or checkpoint cannot serve `LightClientBootstrap` or historical `LightClientUpdates` because it never generated or stored the proofs. + +Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspired by the LC sync strategy: + +1. **Make Lighthouse "collect" historical light client data** — The consensus spec tests verify that a client can construct the full sequence of light client objects, but Lighthouse currently throws away historical data. We need to ensure Lighthouse can generate and persist: + - `LightClientUpdate` for every sync committee period + - `sync_committee_branch` for finalized checkpoint blocks + - `LightClientBootstrap` data for historical finalized blocks + +2. **Design the backfill API** — Once nodes have the data, we need a way to request it. This is the first TBD endpoint and it should look something like: + + ``` + GET /eth/v1/beacon/light_client/updates/backfill?from_period={u64}&count={u64} + ``` + + Or a libp2p protocol. The response would be a batch of `LightClientUpdates`. Key design questions: + - Should this be a REST API (Beacon-API) or a libp2p gossip/rpc protocol? + - How do you rate-limit it? (Historical updates could be megabytes) + - How does a peer prove it has the data vs. lying? + +3. **Pass the spec tests** — There are test vectors in the consensus-specs repo. The goal is to make Lighthouse generate the expected outputs for all historical periods, not just the current one. + +## Specification + +### Phase 1: Historical Light Client Data Collection in Lighthouse + +**The Problem:** `recompute_and_cache_updates()` in `LightClientServerCache` already computes Merkle proofs, constructs `LightClientUpdates`, and stores `SyncCommitteeBranches` to the database. However, it is only invoked for recent blocks because `import_block_update_metrics_and_events()` gates the light client server channel as explained earlier. Furthermore, `cache_state_data()` runs for all blocks but only persists to an in-memory LRU cache of size 32, meaning proofs are computed and then discarded for historical blocks. + +**The Solution:** Implement a post-sync backfill task that walks the finalized chain from the Altair fork to the present and reuses the existing `recompute_and_cache_updates()` logic to populate the database for all historical sync committee periods. + +- **Trigger:** After the node transitions from `SyncState::Syncing` to `SyncState::Synced`, spawn a background task (`lc_backfill`) that iterates finalized blocks period-by-period. +- **Resumability:** Before processing a period, check `store.get_light_client_update(period)`. If it exists, skip. +- **Optimization:** Rather than loading the state for every block in a period (~8,192 slots), iterate block headers via `get_blinded_block()`, identify the block with the best `SyncAggregate` participation in that period, and only load the state and call `recompute_and_cache_updates()` for that candidate. This reduces expensive state loads from thousands per period to one per period. +- **Bootstrap Data:** Ensure `store_sync_committee_branch()` and `store_sync_committee()` are also called during backfill so that `get_light_client_bootstrap()` works for any historical finalized checkpoint, not just recent ones. +- **Fallback for On-Demand Generation:** For nodes that cannot run the full backfill (e.g., due to pruned states), modify `get_light_client_bootstrap()` and `get_light_client_updates()` to fall back to on-demand computation: if the DB entry is missing, load the archived state from the freezer DB, compute the proof or update, store it, and return it. The pattern for this already exists in `get_or_compute_prev_block_cache()`. +- **Testing:** Ensure Lighthouse passes the consensus-specs light client data collection tests. Currently, Lighthouse lacks the data collection coverage because it cannot reconstruct the full historical sequence. + +**Data Collection Test Handler (Aarish)** + +The `light_client_data_collection` test handler has been implemented in `testing/ef_tests/src/cases/light_client_data_collection.rs`. The handler follows Lighthouse's `LoadCase` + `Case` trait pattern: + +- `LoadCase` reads `initial_state.ssz_snappy` into a Beacon state, parses `steps.yaml` using typed structs, and loads `SignedBeaconBlock` objects using fork-aware SSZ deserialization via `from_ssz_bytes_by_fork` +- `Case` initializes a `BeaconChainHarness` from the initial state, processes `NewBlock` steps by importing blocks and calling `recompute_and_cache_updates` directly on `light_client_server_cache` using the block's own sync aggregate +- `NewHead` step handling is in progress — will verify the light client cache against expected values loaded from SSZ files +- Draft PR: [sigp/lighthouse#9666](https://github.com/sigp/lighthouse/pull/9666) + +### Phase 2: LightClientBeaconSnapshot Endpoint + +Design and prototype an endpoint to serve a recent, agreed-upon state root with a Merkle proof connecting it to the `beacon_block_root` that the light client already trusts. + +The snapshot contains: `beacon_block_root`, state root, and a Merkle proof that state root is the correct field in the `BeaconBlockHeader`. Since the block header is already trustlessly known from LC sync, the proof allows the node to verify the state root without trusting the endpoint provider. + +This requires nodes to collect and serve the snapshot data, and likely requires an addition to the Beacon API or a new libp2p protocol. + +### Phase 3: Beacon Sync Protocol — State Chunking + +Design the protocol for fetching the `BeaconState` in verifiable chunks. The proposed approach is: + +- Fixed-size chunks (e.g., 256KB) rather than logical field boundaries. The beacon state is dozens of MB — not GBs — so it can be transferred, but not in a single chunk that would monopolize bandwidth. +- Each chunk carries a Merkle multi-proof against the state root, allowing independent verification. +- Parallel fetching from multiple peers, with immediate rejection and re-request of any chunk that fails verification. +- The chunking strategy should align with SSZ's natural tree layout to avoid extra hashing. + +### Phase 4: Integration & End-to-End Testing + +- Wire the components together: LC bootstrap, forward sync, snapshot, chunk fetch, state assembly. +- Test that a new peer can join the network, validate cryptographically all the way to the present, and transition to full node duties without trusting a checkpoint URL. +- Write spec tests and integration tests for the backfill API. + +## Roadmap + +| Phase | Timeline | Deliverables | +|-------|----------|--------------| +| Phase 1a | Week 7 – 8 | Implement post-sync backfill task; remove historical data gaps in `LightClientUpdate` and `SyncCommitteeBranch` storage | +| Phase 1b | Week 8 – 10 | Implement on-demand fallback for `get_light_client_bootstrap` and `get_light_client_updates`; pass consensus-specs data collection tests | +| Phase 2 | Week 10 – 12 | Design and prototype `LightClientBeaconSnapshot` endpoint (state root + Merkle proof) | +| Phase 3 | Week 12 – 14 | Design beacon sync chunking protocol (fixed-size chunks with Merkle multi-proofs); prototype p2p endpoint | +| Phase 4 | Week 14 – 16 | Integration testing: end-to-end trustless checkpoint sync; spec test compliance; documentation | + +## Possible challenges + +- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. If a node pruned states before backfill completed, or if it checkpoint-synced and never had old states, it cannot generate historical light client data. We must document that backfill requires either archive node configuration or fetching missing data from peers (the backfill API). +- **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. +- **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. +- **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking. +- **EIP-7658 Scope:** EIP-7658 defines a hard-fork mechanism for trustlessly proving that a served `LightClientUpdate` is the canonical "best" one. This project operates before such a hard fork, meaning peers must trust that the update served is the best available. We must be clear about this trust assumption and design the protocol to be forward-compatible with EIP-7658. + +## Goal of the project + +**Minimum Viable Goal** + +- Lighthouse generates, stores, and serves historical `LightClientUpdates` for all sync committee periods from Altair to present. +- Lighthouse stores `SyncCommitteeBranch` and `SyncCommittee` for all historical finalized checkpoints, enabling `get_light_client_bootstrap` for any valid block root. +- Lighthouse passes all consensus-specs light client data collection tests. + +**Stretch Goals** + +- A working `LightClientBeaconSnapshot` endpoint that serves a recent state root with a Merkle proof against a trustlessly known block header. +- A working pathway for nodes to fetch and verify the `BeaconState` in fixed-size chunks with Merkle proofs, completing the trustless checkpoint sync loop. +- A documented backfill API (p2p or REST) allowing nodes to discover and fetch missing historical light client updates from peers. + +**Success Criteria** + +This project will be considered successful when a new peer can join the Ethereum consensus network without trusting a checkpoint URL, using only: + +- A hardcoded, network-verified block root +- The light client sync protocol +- Cryptographically verifiable proofs of state inclusion +- The standard p2p network for data availability + +## Collaborators + +### Fellows + +- [Roheemah](https://github.com/AbolareRoheemah) +- [Aarish](https://github.com/aarishnaiyer) +- [Yee](https://github.com/yxz252426) + +### Mentors + +- [Etan](https://github.com/etan-status) + +## Resources + +- [Aarish's draft PR — Data collection test handler](https://github.com/sigp/lighthouse/pull/9666) +- [Altair light client sync protocol](https://github.com/ethereum/consensus-specs/blob/master/specs/altair/light-client/sync-protocol.md) +- [Data collection test format](https://github.com/ethereum/consensus-specs/blob/master/tests/formats/light_client/data_collection.md) +- [Etan's Nimbus implementation reference](https://github.com/status-im/nimbus-eth2/blob/stable/tests/consensus_spec/test_fixture_light_client_data_collection.nim) +- [Lighthouse codebase](https://github.com/sigp/lighthouse) +- [Beacon API — getLightClientBootstrap](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientBootstrap) +- [Beacon API — getLightClientUpdatesByRange](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientUpdatesByRange) +- [Beacon API — getLightClientFinalityUpdate](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientFinalityUpdate) +- [Beacon API — getLightClientOptimisticUpdate](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientOptimisticUpdate) +- [Beacon API — event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream) From 9f0c65669919a24cd8c8f7239b30fef7cce7e7cc Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Mon, 3 Aug 2026 23:46:48 +0100 Subject: [PATCH 2/8] changed chunking strategy, assigned fellows to tasks --- ...ighthouse-decentralized-checkpoint-sync.md | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index d922aec..62ca2cf 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -114,8 +114,7 @@ This requires nodes to collect and serve the snapshot data, and likely requires Design the protocol for fetching the `BeaconState` in verifiable chunks. The proposed approach is: -- Fixed-size chunks (e.g., 256KB) rather than logical field boundaries. The beacon state is dozens of MB — not GBs — so it can be transferred, but not in a single chunk that would monopolize bandwidth. -- Each chunk carries a Merkle multi-proof against the state root, allowing independent verification. +- Chunk by top-level `BeaconState` fields (and possibly sub-chunk large fields like `validators` by index range). This aligns with SSZ's natural Merkle tree structure, allowing each chunk to be verified with standard Merkle proofs against known generalized indices. - Parallel fetching from multiple peers, with immediate rejection and re-request of any chunk that fails verification. - The chunking strategy should align with SSZ's natural tree layout to avoid extra hashing. @@ -127,13 +126,17 @@ Design the protocol for fetching the `BeaconState` in verifiable chunks. The pro ## Roadmap -| Phase | Timeline | Deliverables | -|-------|----------|--------------| -| Phase 1a | Week 7 – 8 | Implement post-sync backfill task; remove historical data gaps in `LightClientUpdate` and `SyncCommitteeBranch` storage | -| Phase 1b | Week 8 – 10 | Implement on-demand fallback for `get_light_client_bootstrap` and `get_light_client_updates`; pass consensus-specs data collection tests | -| Phase 2 | Week 10 – 12 | Design and prototype `LightClientBeaconSnapshot` endpoint (state root + Merkle proof) | -| Phase 3 | Week 12 – 14 | Design beacon sync chunking protocol (fixed-size chunks with Merkle multi-proofs); prototype p2p endpoint | -| Phase 4 | Week 14 – 16 | Integration testing: end-to-end trustless checkpoint sync; spec test compliance; documentation | + +| Phase | Timeline | Deliverables | Fellow(s) Responsible | +| -------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | +| Phase 1a | Week 7 – 8 | Implement post-sync backfill task; remove historical data gaps in `LightClientUpdate` and `SyncCommitteeBranch` storage | Roheemah | +| Phase 1b | Week 8 – 11 | Implement on-demand fallback for `get_light_client_bootstrap` and `get_light_client_updates`; pass consensus-specs data collection tests | Roheemah, Aarish | +| Phase 2 | Week 10 – 12 | Design and prototype `LightClientBeaconSnapshot` endpoint (state root + Merkle proof) | Yee | +| Phase 3 | Week 12 – 14 | Design beacon sync chunking protocol (fixed-size chunks with Merkle multi-proofs); prototype p2p endpoint | Aarish | +| Phase 4 | Week 14 – 16 | Integration testing: end-to-end trustless checkpoint sync; spec test compliance; documentation | Roheemah, Aarish, Yee | + + + ## Possible challenges From 13b890d6e6731905bdb838c1a2130349d28cff79 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Tue, 25 Aug 2026 12:05:32 +0100 Subject: [PATCH 3/8] replaced partly wrong code under recency gap, reworded phase1 and added the backfill endpoint suggested by Etan --- ...ighthouse-decentralized-checkpoint-sync.md | 29 ++++++++++++------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index 62ca2cf..d422a75 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -25,14 +25,21 @@ The core of this project, however, is not just the high-level flow — it is the **Recency guard in `import_block_update_metrics_and_events`:** ```rust -// Do not write to the cache for blocks older than 2 epochs, this helps reduce writes to -// the cache during sync. -if block_delay_total < self.slot_clock.slot_duration() * 64 { - // Store the timestamp of the block being imported into the cache. - self.block_times_cache.write().set_time_imported( - block_root, - current_slot, - block_time_imported, +// Do not trigger light_client server update producer for old blocks, to extra work +// during sync. +if self.config.enable_light_client_server + && block_delay_total < self.slot_clock.slot_duration() * 32 + && let Some(mut light_client_server_tx) = self.light_client_server_tx.clone() + && let Ok(sync_aggregate) = block.body().sync_aggregate() + && let Err(e) = light_client_server_tx.try_send(( + block.parent_root(), + block.slot(), + sync_aggregate.clone(), + )) +{ + warn!( + error = ?e, + "Failed to send light_client server event" ); } ``` @@ -84,13 +91,15 @@ Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspire **The Problem:** `recompute_and_cache_updates()` in `LightClientServerCache` already computes Merkle proofs, constructs `LightClientUpdates`, and stores `SyncCommitteeBranches` to the database. However, it is only invoked for recent blocks because `import_block_update_metrics_and_events()` gates the light client server channel as explained earlier. Furthermore, `cache_state_data()` runs for all blocks but only persists to an in-memory LRU cache of size 32, meaning proofs are computed and then discarded for historical blocks. -**The Solution:** Implement a post-sync backfill task that walks the finalized chain from the Altair fork to the present and reuses the existing `recompute_and_cache_updates()` logic to populate the database for all historical sync committee periods. + +**The Solution:** Implement a post-sync backfill task that walks the finalized chain from the Altair fork to the present. For each sync committee period, check if LightClientUpdate and SyncCommitteeBranch data exists in the DB. If not, load the corresponding state (from DB) and call `recompute_and_cache_updates`. For pre-checkpoint data, nodes must rely on the network backfill API (to be defined). - **Trigger:** After the node transitions from `SyncState::Syncing` to `SyncState::Synced`, spawn a background task (`lc_backfill`) that iterates finalized blocks period-by-period. - **Resumability:** Before processing a period, check `store.get_light_client_update(period)`. If it exists, skip. - **Optimization:** Rather than loading the state for every block in a period (~8,192 slots), iterate block headers via `get_blinded_block()`, identify the block with the best `SyncAggregate` participation in that period, and only load the state and call `recompute_and_cache_updates()` for that candidate. This reduces expensive state loads from thousands per period to one per period. - **Bootstrap Data:** Ensure `store_sync_committee_branch()` and `store_sync_committee()` are also called during backfill so that `get_light_client_bootstrap()` works for any historical finalized checkpoint, not just recent ones. -- **Fallback for On-Demand Generation:** For nodes that cannot run the full backfill (e.g., due to pruned states), modify `get_light_client_bootstrap()` and `get_light_client_updates()` to fall back to on-demand computation: if the DB entry is missing, load the archived state from the freezer DB, compute the proof or update, store it, and return it. The pattern for this already exists in `get_or_compute_prev_block_cache()`. +- **Fallback for On-Demand Generation:** For nodes that cannot run the full backfill (e.g., due to pruned states), modify `get_light_client_bootstrap()` and `get_light_client_updates()` to fall back to on-demand computation: if the DB entry is missing, load the archived state from the freezer DB, compute the proof or update, store it, and return it. The pattern for this already exists in `get_or_compute_prev_block_cache()`. This fallback only succeeds if the node actually has the corresponding BeaconState. For checkpoint-synced nodes, pre-checkpoint requests will fail locally but can be served via a network backfill API. +- **Network Backfill API:** Define and implement a new p2p endpoint allowing nodes to fetch historical LightClientUpdates and SyncCommitteeBranches from peers who were online during the requested periods and have stored the data locally. This is the only path to obtain data from before the node's own checkpoint, since local generation is impossible without the state. - **Testing:** Ensure Lighthouse passes the consensus-specs light client data collection tests. Currently, Lighthouse lacks the data collection coverage because it cannot reconstruct the full historical sequence. **Data Collection Test Handler (Aarish)** From a4a2baf50bf358f481d2e8a24eed40dd6f9662c4 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Mon, 31 Aug 2026 22:29:51 +0100 Subject: [PATCH 4/8] updated proposal based on Etan's new documentation --- ...ighthouse-decentralized-checkpoint-sync.md | 103 +++++++++++------- 1 file changed, 66 insertions(+), 37 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index d422a75..63d0a1a 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -12,7 +12,7 @@ This work primarily affects the sync protocol of full nodes and extends the resp ## Project description -Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, inspired by the light client (LC) sync protocol and guided by ongoing work on light client data backfill ([EIP-7658](https://eips.ethereum.org/EIPS/eip-7658)). The flow for a joining node is: +Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, inspired by the light client (LC) sync protocol and guided by Etan's [decentralized CL sync specification](https://hackmd.io/@etan-status/decentralized-cl-sync), which supersedes the earlier EIP-7658 approach by eliminating the hard-fork requirement and defining concrete p2p endpoints for epoch-level backfill and state snap sync. The flow for a joining node is: 1. **Light Client Bootstrap:** The node starts with a trusted block root baked into the client (e.g., the first Altair block root). It requests a `LightClientBootstrap` and initializes a `LightClientStore`. 2. **Forward Sync:** The node requests `LightClientUpdates` by range, syncing forward from the trusted root to the present. It now has a verified recent `beacon_block_root`. @@ -87,45 +87,74 @@ Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspire ## Specification -### Phase 1: Historical Light Client Data Collection in Lighthouse +### Phase 1: Historical Light Client Epoch Data Collection & Serving -**The Problem:** `recompute_and_cache_updates()` in `LightClientServerCache` already computes Merkle proofs, constructs `LightClientUpdates`, and stores `SyncCommitteeBranches` to the database. However, it is only invoked for recent blocks because `import_block_update_metrics_and_events()` gates the light client server channel as explained earlier. Furthermore, `cache_state_data()` runs for all blocks but only persists to an in-memory LRU cache of size 32, meaning proofs are computed and then discarded for historical blocks. +**The Problem:** Lighthouse already computes light client proofs for every block during sync, but the recency guard in `import_block_update_metrics_and_events` and the bounded channel (`LIGHT_CLIENT_SERVER_CHANNEL_CAPACITY = 32`) prevent historical data from reaching the database. Additionally, `get_light_client_bootstrap` explicitly lacks a backfill mechanism. The result: checkpoint-synced nodes cannot serve historical LC data to peers. +**The Solution (Local Collection — Phase 1a):** Implement a post-sync backfill task that walks the finalized chain from the node's **earliest available state** to the present. For each sync committee period, identify the best block (highest sync aggregate participation), load its state from the freezer DB, and call `recompute_and_cache_updates` to store the canonical `LightClientUpdate` and `SyncCommitteeBranch`. -**The Solution:** Implement a post-sync backfill task that walks the finalized chain from the Altair fork to the present. For each sync committee period, check if LightClientUpdate and SyncCommitteeBranch data exists in the DB. If not, load the corresponding state (from DB) and call `recompute_and_cache_updates`. For pre-checkpoint data, nodes must rely on the network backfill API (to be defined). +This task: +- Spawns when `BackFillState::Completed` is set +- Is resumable (checks `store.get_light_client_update(period)` before processing) +- Is pausable and low-priority (yields to validator duties) +- Only works for periods where the node has the `BeaconState` -- **Trigger:** After the node transitions from `SyncState::Syncing` to `SyncState::Synced`, spawn a background task (`lc_backfill`) that iterates finalized blocks period-by-period. -- **Resumability:** Before processing a period, check `store.get_light_client_update(period)`. If it exists, skip. -- **Optimization:** Rather than loading the state for every block in a period (~8,192 slots), iterate block headers via `get_blinded_block()`, identify the block with the best `SyncAggregate` participation in that period, and only load the state and call `recompute_and_cache_updates()` for that candidate. This reduces expensive state loads from thousands per period to one per period. -- **Bootstrap Data:** Ensure `store_sync_committee_branch()` and `store_sync_committee()` are also called during backfill so that `get_light_client_bootstrap()` works for any historical finalized checkpoint, not just recent ones. -- **Fallback for On-Demand Generation:** For nodes that cannot run the full backfill (e.g., due to pruned states), modify `get_light_client_bootstrap()` and `get_light_client_updates()` to fall back to on-demand computation: if the DB entry is missing, load the archived state from the freezer DB, compute the proof or update, store it, and return it. The pattern for this already exists in `get_or_compute_prev_block_cache()`. This fallback only succeeds if the node actually has the corresponding BeaconState. For checkpoint-synced nodes, pre-checkpoint requests will fail locally but can be served via a network backfill API. -- **Network Backfill API:** Define and implement a new p2p endpoint allowing nodes to fetch historical LightClientUpdates and SyncCommitteeBranches from peers who were online during the requested periods and have stored the data locally. This is the only path to obtain data from before the node's own checkpoint, since local generation is impossible without the state. -- **Testing:** Ensure Lighthouse passes the consensus-specs light client data collection tests. Currently, Lighthouse lacks the data collection coverage because it cannot reconstruct the full historical sequence. +**The Solution (P2P Serving — Phase 1b):** Implement the `LightClientDataBackfillByRange` libp2p endpoint as specified in Etan's draft: -**Data Collection Test Handler (Aarish)** +/eth2/beacon_chain/req/light_client_data_backfill_by_range/0/ -The `light_client_data_collection` test handler has been implemented in `testing/ef_tests/src/cases/light_client_data_collection.rs`. The handler follows Lighthouse's `LoadCase` + `Case` trait pattern: +Request: (start_epoch: Epoch, count: uint64) +Response: List[LightClientEpochData, MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA] +MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA := 256 -- `LoadCase` reads `initial_state.ssz_snappy` into a Beacon state, parses `steps.yaml` using typed structs, and loads `SignedBeaconBlock` objects using fork-aware SSZ deserialization via `from_ssz_bytes_by_fork` -- `Case` initializes a `BeaconChainHarness` from the initial state, processes `NewBlock` steps by importing blocks and calling `recompute_and_cache_updates` directly on `light_client_server_cache` using the block's own sync aggregate -- `NewHead` step handling is in progress — will verify the light client cache against expected values loaded from SSZ files -- Draft PR: [sigp/lighthouse#9666](https://github.com/sigp/lighthouse/pull/9666) +`LightClientEpochData` contains per-epoch raw block data including `sync_committee_bits`, `sync_aggregate_branch`, `finalized_checkpoint`, and `current_sync_committee` — everything a receiver needs to independently simulate `is_better_update` and verify the canonical best update for a period. -### Phase 2: LightClientBeaconSnapshot Endpoint +Key constraints: +- Only serves finalized data +- Fork context determined from last non-empty `block_data[i]` (or `epoch` if all empty) +- Rate-limited to 256 epochs per request -Design and prototype an endpoint to serve a recent, agreed-upon state root with a Merkle proof connecting it to the `beacon_block_root` that the light client already trusts. +**On-Demand Fallback:** For periods not yet backfilled, modify `get_light_client_updates` and `get_light_client_bootstrap` to check the DB first, then fall back to computing from archived states if available. For pre-checkpoint data, the node must request from peers via the endpoint above — local generation is impossible without the state. -The snapshot contains: `beacon_block_root`, state root, and a Merkle proof that state root is the correct field in the `BeaconBlockHeader`. Since the block header is already trustlessly known from LC sync, the proof allows the node to verify the state root without trusting the endpoint provider. +**Testing:** Pass consensus-specs light client data collection tests. Aarish's draft PR (#9666) implements the test handler. -This requires nodes to collect and serve the snapshot data, and likely requires an addition to the Beacon API or a new libp2p protocol. +### Phase 2: BeaconStateSnapshot & Checkpoint Bootstrap (Yee) -### Phase 3: Beacon Sync Protocol — State Chunking +Implement the `BeaconStateSnapshot` endpoint for state snap sync: -Design the protocol for fetching the `BeaconState` in verifiable chunks. The proposed approach is: +/eth2/beacon_chain/req/beacon_state_summary/0/ +Request: (block_root: Root) # LightClientStore.finalized_header +Response: BeaconStateSnapshot -- Chunk by top-level `BeaconState` fields (and possibly sub-chunk large fields like `validators` by index range). This aligns with SSZ's natural Merkle tree structure, allowing each chunk to be verified with standard Merkle proofs against known generalized indices. -- Parallel fetching from multiple peers, with immediate rejection and re-request of any chunk that fails verification. -- The chunking strategy should align with SSZ's natural tree layout to avoid extra hashing. +The `BeaconStateSnapshot` contains: +- `summary: BeaconStateSummary` — a mirror of `BeaconState` where all `List` / `ProgressiveList` fields are summarized as `ListSummary { items_root, num_items }`, preserving the same `hash_tree_root` +- `state_branch: ProgressiveList[Bytes32]` — Merkle proof for the summary + +This allows a light client to obtain a compact, verifiable summary of the `BeaconState` at the start of a sync period, from which it can then request individual state parts. + +The server must keep the last 2 summaries available to avoid rollover during ongoing downloads. + +Also implement the trusted checkpoint bootstrap mechanism: if network metadata contains `trusted_checkpoint.txt` with `0x:`, light clients start syncing from this root; otherwise, use genesis (if post-Altair) or require `--trusted-block-root`. + +### Phase 3: State Snap Sync - BeaconStatePartsByRange (Aarish) + +Implement the state chunking protocol as specified: + +/eth2/beacon_chain/req/beacon_state_parts_by_range/0/ +Request: (start_chunk: uint64, count: uint64) +Response: List[BeaconStatePart, MAX_REQUEST_BEACON_STATE_PARTS] +MAX_REQUEST_BEACON_STATE_PARTS := 16 + +`BeaconStatePart` contains: +- `chunk_index: uint64` +- `data: ProgressiveByteList` — the actual chunk +- `branch: ProgressiveList[Bytes32]` — Merkle proof + +Chunking is deterministic per chunk ID based on `ListSummary` fields. Each list has a defined "items per chunk" (e.g., validators: 2^12 per chunk, balances: 2^16 per chunk). Target: <0.5 MB per chunk. + +The first chunk of a `ProgressiveList` contains all smaller subtrees that fit completely within the items-per-chunk budget. For example, if items-per-chunk is 32, the first chunk contains 1+4+16 = 21 items, and subsequent chunks contain 32 items each. + +The node fetches the `BeaconStateSnapshot` first (Phase 2), then requests parts by range, verifies each chunk's Merkle proof against the summary, and reassembles the full state. ### Phase 4: Integration & End-to-End Testing @@ -135,25 +164,23 @@ Design the protocol for fetching the `BeaconState` in verifiable chunks. The pro ## Roadmap - -| Phase | Timeline | Deliverables | Fellow(s) Responsible | -| -------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| Phase 1a | Week 7 – 8 | Implement post-sync backfill task; remove historical data gaps in `LightClientUpdate` and `SyncCommitteeBranch` storage | Roheemah | -| Phase 1b | Week 8 – 11 | Implement on-demand fallback for `get_light_client_bootstrap` and `get_light_client_updates`; pass consensus-specs data collection tests | Roheemah, Aarish | -| Phase 2 | Week 10 – 12 | Design and prototype `LightClientBeaconSnapshot` endpoint (state root + Merkle proof) | Yee | -| Phase 3 | Week 12 – 14 | Design beacon sync chunking protocol (fixed-size chunks with Merkle multi-proofs); prototype p2p endpoint | Aarish | -| Phase 4 | Week 14 – 16 | Integration testing: end-to-end trustless checkpoint sync; spec test compliance; documentation | Roheemah, Aarish, Yee | - - +| Phase | Timeline | Deliverables | Fellow(s) | +| -------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------ | ---------------- | +| Phase 1a | Week 7 – 9 | Post-sync backfill task; store historical `LightClientUpdate` + `SyncCommitteeBranch` from earliest available state to present | Roheemah | +| Phase 1b | Week 9 – 12 | `LightClientDataBackfillByRange` p2p endpoint; on-demand fallback; pass consensus-specs data collection tests | Roheemah, Aarish | +| Phase 2 | Week 12 – 14 | `BeaconStateSnapshot` endpoint; trusted checkpoint bootstrap | Yee | +| Phase 3 | Week 14 – 15 | `BeaconStatePartsByRange` endpoint; deterministic chunking | Aarish | +| Phase 4 | Week 15 – 16 | End-to-end integration; documentation | All | ## Possible challenges +- **Protocol evolution risk:** Etan's HackMD spec is explicitly marked as a draft ("TBD and likely not yet optimal"). The LightClientEpochData container, chunk sizes, and endpoint paths may change during implementation. - **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. If a node pruned states before backfill completed, or if it checkpoint-synced and never had old states, it cannot generate historical light client data. We must document that backfill requires either archive node configuration or fetching missing data from peers (the backfill API). - **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. - **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. - **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking. -- **EIP-7658 Scope:** EIP-7658 defines a hard-fork mechanism for trustlessly proving that a served `LightClientUpdate` is the canonical "best" one. This project operates before such a hard fork, meaning peers must trust that the update served is the best available. We must be clear about this trust assumption and design the protocol to be forward-compatible with EIP-7658. +- Etan's revised spec avoids BeaconState modifications entirely. The trust model now relies on Merkle-proved LightClientEpochData rather than enshrined state tracking. ## Goal of the project @@ -192,6 +219,8 @@ This project will be considered successful when a new peer can join the Ethereum ## Resources +- [Etan's decentralized CL sync spec](https://hackmd.io/@etan-status/decentralized-cl-sync) +- [Nimbus PR #8445 — historical LC backfill design](https://github.com/status-im/nimbus-eth2/pull/8445) - [Aarish's draft PR — Data collection test handler](https://github.com/sigp/lighthouse/pull/9666) - [Altair light client sync protocol](https://github.com/ethereum/consensus-specs/blob/master/specs/altair/light-client/sync-protocol.md) - [Data collection test format](https://github.com/ethereum/consensus-specs/blob/master/tests/formats/light_client/data_collection.md) From 39fb2627506747e0b0dff02ae99749d928cad847 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Wed, 2 Sep 2026 12:45:53 +0100 Subject: [PATCH 5/8] add more details and testing information, majorly to phase 1 --- ...ighthouse-decentralized-checkpoint-sync.md | 108 ++++++++++++++---- 1 file changed, 84 insertions(+), 24 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index 63d0a1a..6eb523d 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -67,23 +67,17 @@ The combined effect: a Lighthouse node that synced from genesis or checkpoint ca Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspired by the LC sync strategy: -1. **Make Lighthouse "collect" historical light client data** — The consensus spec tests verify that a client can construct the full sequence of light client objects, but Lighthouse currently throws away historical data. We need to ensure Lighthouse can generate and persist: +1. **Make Lighthouse "collect" historical light client data** — The consensus spec tests verify that a client can construct the full sequence of light client objects, but Lighthouse currently throws away historical data. We need to ensure Lighthouse can generate and persist, for every period it has state available: - `LightClientUpdate` for every sync committee period - `sync_committee_branch` for finalized checkpoint blocks - `LightClientBootstrap` data for historical finalized blocks -2. **Design the backfill API** — Once nodes have the data, we need a way to request it. This is the first TBD endpoint and it should look something like: +2. **Design the backfill API** — Once nodes have the data, we need a way to request it from peers for periods they don't have locally. This is a libp2p request/response protocol, following the pattern of Etan's `LightClientDataBackfillByRange` spec. Key design questions already resolved by the spec: + - Transport: libp2p req/resp + - Rate limiting: capped at 256 epochs per request + - Proof-of-honesty: every field in the response is independently Merkle-verifiable by the requester, so a lying peer is detectable, not merely trusted - ``` - GET /eth/v1/beacon/light_client/updates/backfill?from_period={u64}&count={u64} - ``` - - Or a libp2p protocol. The response would be a batch of `LightClientUpdates`. Key design questions: - - Should this be a REST API (Beacon-API) or a libp2p gossip/rpc protocol? - - How do you rate-limit it? (Historical updates could be megabytes) - - How does a peer prove it has the data vs. lying? - -3. **Pass the spec tests** — There are test vectors in the consensus-specs repo. The goal is to make Lighthouse generate the expected outputs for all historical periods, not just the current one. +3. **Pass the spec tests** — There are test vectors in the consensus-specs repo. The goal is to make Lighthouse generate the expected outputs for all periods it has state for, not just the current one. ## Specification @@ -91,21 +85,29 @@ Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspire **The Problem:** Lighthouse already computes light client proofs for every block during sync, but the recency guard in `import_block_update_metrics_and_events` and the bounded channel (`LIGHT_CLIENT_SERVER_CHANNEL_CAPACITY = 32`) prevent historical data from reaching the database. Additionally, `get_light_client_bootstrap` explicitly lacks a backfill mechanism. The result: checkpoint-synced nodes cannot serve historical LC data to peers. -**The Solution (Local Collection — Phase 1a):** Implement a post-sync backfill task that walks the finalized chain from the node's **earliest available state** to the present. For each sync committee period, identify the best block (highest sync aggregate participation), load its state from the freezer DB, and call `recompute_and_cache_updates` to store the canonical `LightClientUpdate` and `SyncCommitteeBranch`. +**The Solution (Local Collection — Phase 1a):** Implement a post-sync backfill task that walks the finalized chain from the node's **earliest available state** to the present. (Note: this is bounded by whichever `BeaconState`s the node actually retains — a checkpoint-synced, non-archive node cannot walk back to Altair; only an archive node can. Backfill coverage is therefore a function of node configuration, not a project guarantee of full Altair-to-present coverage on every node type.) For each sync committee period, identify the best block (highest sync aggregate participation), load its state from the freezer DB, and call `recompute_and_cache_updates` to store the canonical `LightClientUpdate` and `SyncCommitteeBranch`. This task: - Spawns when `BackFillState::Completed` is set - Is resumable (checks `store.get_light_client_update(period)` before processing) - Is pausable and low-priority (yields to validator duties) - Only works for periods where the node has the `BeaconState` +- Reuses Lighthouse's **existing** `DBColumn::LightClientUpdate` / `SyncCommitteeBranch` / `SyncCommittee` storage — this phase does not introduce a new DB column for `LightClientEpochData`. `LightClientEpochData` is a **wire/transport container** defined by Etan's spec for the p2p endpoint below; it is assembled on demand from the existing stored fields when serving a request, not persisted as its own schema. Etan's spec is explicitly marked as a draft ("TBD and likely not yet optimal"), so we avoid committing local storage to a container shape that may still change. **The Solution (P2P Serving — Phase 1b):** Implement the `LightClientDataBackfillByRange` libp2p endpoint as specified in Etan's draft: +``` /eth2/beacon_chain/req/light_client_data_backfill_by_range/0/ Request: (start_epoch: Epoch, count: uint64) Response: List[LightClientEpochData, MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA] MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA := 256 +``` + +Implementation follows the pattern of the existing `handle_light_client_updates_by_range` handler in `network_beacon_processor::rpc_methods.rs`: +- Validate the request (count ≤ 256, `start_epoch` and range must be finalized) +- Query stored `LightClientUpdate`/`SyncCommitteeBranch` data for the requested range and assemble each `LightClientEpochData` response entry +- Stream responses back via `SendResponse` `LightClientEpochData` contains per-epoch raw block data including `sync_committee_bits`, `sync_aggregate_branch`, `finalized_checkpoint`, and `current_sync_committee` — everything a receiver needs to independently simulate `is_better_update` and verify the canonical best update for a period. @@ -114,17 +116,45 @@ Key constraints: - Fork context determined from last non-empty `block_data[i]` (or `epoch` if all empty) - Rate-limited to 256 epochs per request -**On-Demand Fallback:** For periods not yet backfilled, modify `get_light_client_updates` and `get_light_client_bootstrap` to check the DB first, then fall back to computing from archived states if available. For pre-checkpoint data, the node must request from peers via the endpoint above — local generation is impossible without the state. +**On-Demand Fallback:** For periods not yet backfilled, modify `get_light_client_updates` and `get_light_client_bootstrap` to check the DB first, then fall back to computing from archived states if available (archive-mode nodes only). For pre-checkpoint data on a non-archive node, local generation is impossible — the node must request it from peers via the endpoint above. + +**Testing (Phase 1a/1b):** -**Testing:** Pass consensus-specs light client data collection tests. Aarish's draft PR (#9666) implements the test handler. +1. **Unit tests** + - `recompute_and_cache_updates` produces a correct `LightClientUpdate` and `SyncCommitteeBranch` for a known-good historical period + - Constructing `LightClientEpochData` from stored `LightClientUpdate`/`SyncCommitteeBranch` data round-trips correctly against Etan's container spec + - Best-candidate selection (highest sync aggregate participation) picked correctly across a period with missed slots + +2. **Integration tests** + - Post-sync backfill completes for all periods from earliest available state to present + - Resume-on-crash: backfill continues from the last completed period rather than restarting + - `get_light_client_bootstrap` succeeds for any historical finalized checkpoint the node has backfilled + - On-demand fallback returns correct data for archive-mode nodes and correctly errors (or falls through to network) for non-archive nodes + +3. **Consensus spec tests** + - Pass `light_client_data_collection` test vectors (Aarish's draft PR #9666 implements the test handler) + +4. **P2P endpoint tests** + - Peer requests an epoch range and receives the expected `LightClientEpochData` list + - Requests over `MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA` (256) are rejected + - Fork context is correctly derived from the last non-empty `block_data[i]`, falling back to `epoch` when the whole range is empty + - Requests for unfinalized data are rejected + +5. **Adversarial tests** (design in Phase 1, exercise in Phase 4) + - Peer returns incorrect `sync_committee_bits` → rejected on proof mismatch + - Peer omits blocks in a period → detected via Merkle proof verification + - Peer returns unfinalized or malformed data → rejected ### Phase 2: BeaconStateSnapshot & Checkpoint Bootstrap (Yee) Implement the `BeaconStateSnapshot` endpoint for state snap sync: +``` /eth2/beacon_chain/req/beacon_state_summary/0/ + Request: (block_root: Root) # LightClientStore.finalized_header Response: BeaconStateSnapshot +``` The `BeaconStateSnapshot` contains: - `summary: BeaconStateSummary` — a mirror of `BeaconState` where all `List` / `ProgressiveList` fields are summarized as `ListSummary { items_root, num_items }`, preserving the same `hash_tree_root` @@ -136,14 +166,17 @@ The server must keep the last 2 summaries available to avoid rollover during ong Also implement the trusted checkpoint bootstrap mechanism: if network metadata contains `trusted_checkpoint.txt` with `0x:`, light clients start syncing from this root; otherwise, use genesis (if post-Altair) or require `--trusted-block-root`. -### Phase 3: State Snap Sync - BeaconStatePartsByRange (Aarish) +### Phase 3: State Snap Sync — BeaconStatePartsByRange (Aarish) Implement the state chunking protocol as specified: +``` /eth2/beacon_chain/req/beacon_state_parts_by_range/0/ + Request: (start_chunk: uint64, count: uint64) Response: List[BeaconStatePart, MAX_REQUEST_BEACON_STATE_PARTS] MAX_REQUEST_BEACON_STATE_PARTS := 16 +``` `BeaconStatePart` contains: - `chunk_index: uint64` @@ -161,6 +194,34 @@ The node fetches the `BeaconStateSnapshot` first (Phase 2), then requests parts - Wire the components together: LC bootstrap, forward sync, snapshot, chunk fetch, state assembly. - Test that a new peer can join the network, validate cryptographically all the way to the present, and transition to full node duties without trusting a checkpoint URL. - Write spec tests and integration tests for the backfill API. +- Exercise the adversarial test cases designed in Phase 1 against a live testnet peer set. + +## Phase interdependencies + +``` +Phase 1a/1b (Historical LC data) + ├─ Depends on: Altair fork support already in Lighthouse + ├─ Produces: locally verified LightClientUpdate/SyncCommitteeBranch data, + │ served over the network as LightClientEpochData + └─ Used by: Phase 2 (links a verified block header to a state root) + +Phase 2 (BeaconStateSnapshot — Yee) + ├─ Depends on: Phase 1 (a verified recent block header to anchor to) + ├─ Produces: BeaconStateSummary + Merkle proof from block header to state root + └─ Consumed by: Phase 3 (state chunk verification root) + +Phase 3 (BeaconStatePartsByRange — Aarish) + ├─ Depends on: Phase 2 (the state summary chunks are proved against) + ├─ Produces: verifiable BeaconState chunks + └─ Consumed by: Phase 4 (full state reassembly) + +Phase 4 (End-to-end) + ├─ Orchestrates: Phases 1–3 in sequence + ├─ Tests: full checkpoint-sync flow, including adversarial peers + └─ Success: a new node joins without trusting a checkpoint URL +``` + +Phase 1 is independently useful on its own (it fixes a real gap in Lighthouse's existing LC serving today); Phases 2–3 build on it for the full trust-minimized checkpoint-sync pipeline. ## Roadmap @@ -172,29 +233,28 @@ The node fetches the `BeaconStateSnapshot` first (Phase 2), then requests parts | Phase 3 | Week 14 – 15 | `BeaconStatePartsByRange` endpoint; deterministic chunking | Aarish | | Phase 4 | Week 15 – 16 | End-to-end integration; documentation | All | - ## Possible challenges -- **Protocol evolution risk:** Etan's HackMD spec is explicitly marked as a draft ("TBD and likely not yet optimal"). The LightClientEpochData container, chunk sizes, and endpoint paths may change during implementation. -- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. If a node pruned states before backfill completed, or if it checkpoint-synced and never had old states, it cannot generate historical light client data. We must document that backfill requires either archive node configuration or fetching missing data from peers (the backfill API). +- **Protocol evolution risk:** Etan's HackMD spec is explicitly marked as a draft ("TBD and likely not yet optimal"). The `LightClientEpochData` container, chunk sizes, and endpoint paths may change during implementation. We build defensively, pinning only the parts that are stable (the existing `LightClientUpdate` storage, the `is_better_update` ranking logic) and adapting the transport layer as the spec stabilizes. +- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. If a node pruned states before backfill completed, or if it checkpoint-synced and never had old states, it cannot generate historical light client data for those periods. We document that full historical coverage requires either archive node configuration or fetching missing data from peers (the backfill API) — it is not something the local backfill task alone can guarantee on every node type. - **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. - **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. - **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking. -- Etan's revised spec avoids BeaconState modifications entirely. The trust model now relies on Merkle-proved LightClientEpochData rather than enshrined state tracking. +- **No BeaconState modifications required:** Etan's revised spec avoids BeaconState modifications entirely (an improvement over the earlier EIP-7658 approach, which needed one). The trust model relies on Merkle-proved `LightClientEpochData` rather than enshrined state tracking, which is why this project does not require a hard fork. ## Goal of the project **Minimum Viable Goal** -- Lighthouse generates, stores, and serves historical `LightClientUpdates` for all sync committee periods from Altair to present. -- Lighthouse stores `SyncCommitteeBranch` and `SyncCommittee` for all historical finalized checkpoints, enabling `get_light_client_bootstrap` for any valid block root. +- Lighthouse generates, stores, and serves historical `LightClientUpdates` for all sync committee periods it has state available for. +- Lighthouse stores `SyncCommitteeBranch` and `SyncCommittee` for all historical finalized checkpoints it has backfilled, enabling `get_light_client_bootstrap` for those block roots. - Lighthouse passes all consensus-specs light client data collection tests. **Stretch Goals** - A working `LightClientBeaconSnapshot` endpoint that serves a recent state root with a Merkle proof against a trustlessly known block header. - A working pathway for nodes to fetch and verify the `BeaconState` in fixed-size chunks with Merkle proofs, completing the trustless checkpoint sync loop. -- A documented backfill API (p2p or REST) allowing nodes to discover and fetch missing historical light client updates from peers. +- A documented, spec-aligned `LightClientDataBackfillByRange` p2p endpoint allowing nodes to discover and fetch missing historical light client data from peers. **Success Criteria** @@ -230,4 +290,4 @@ This project will be considered successful when a new peer can join the Ethereum - [Beacon API — getLightClientUpdatesByRange](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientUpdatesByRange) - [Beacon API — getLightClientFinalityUpdate](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientFinalityUpdate) - [Beacon API — getLightClientOptimisticUpdate](https://ethereum.github.io/beacon-APIs/#/Beacon/getLightClientOptimisticUpdate) -- [Beacon API — event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream) +- [Beacon API — event stream](https://ethereum.github.io/beacon-APIs/#/Events/eventstream) \ No newline at end of file From a424d659e68aecf0a2162b04e64be84bd64f49ad Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Thu, 3 Sep 2026 23:21:19 +0100 Subject: [PATCH 6/8] updated several sections in Phase 1 --- ...ighthouse-decentralized-checkpoint-sync.md | 71 +++++++++++-------- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index 6d7a993..ecfe3e9 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -12,7 +12,7 @@ This work primarily affects the sync protocol of full nodes and extends the resp ## Project description -Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, inspired by the light client (LC) sync protocol and guided by Etan's [decentralized CL sync specification](https://hackmd.io/@etan-status/decentralized-cl-sync), which supersedes the earlier EIP-7658 approach by eliminating the hard-fork requirement and defining concrete p2p endpoints for epoch-level backfill and state snap sync. The flow for a joining node is: +Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, inspired by the light client (LC) sync protocol and guided by Etan's [decentralized CL sync specification](https://hackmd.io/@etan-status/decentralized-cl-sync), which supersedes the earlier [EIP-7658](https://eips.ethereum.org/EIPS/eip-7658) approach by eliminating the hard-fork requirement and defining concrete p2p endpoints for epoch-level backfill and state snap sync. The flow for a joining node is: 1. **Light Client Bootstrap:** The node starts with a trusted block root baked into the client (e.g., the first Altair block root). It requests a `LightClientBootstrap` and initializes a `LightClientStore`. 2. **Forward Sync:** The node requests `LightClientUpdates` by range, syncing forward from the trusted root to the present. It now has a verified recent `beacon_block_root`. @@ -20,9 +20,9 @@ Our proposed solution is a multi-phase trust-minimized checkpoint sync strategy, 4. **State Backfill:** The node fetches the `BeaconState` at that state root in fixed-size chunks from multiple peers. Each chunk comes with a Merkle proof verifying its inclusion in the state tree. The node verifies each chunk independently and reassembles the full state. 5. **Full Node Activation:** Once the state is fully fetched and verified, the node has everything it needs to transition to full node duties. -The core of this project, however, is not just the high-level flow — it is the Lighthouse client infrastructure required to make this flow possible. Our research into the Lighthouse codebase revealed that the storage schema and serving endpoints for light client data already exist (`DBColumn::LightClientUpdate`, `SyncCommitteeBranch`, `SyncCommittee`, and the p2p `LightClientUpdatesByRange` endpoint). The critical gap is that historical data is not persisted due to the following barriers: +The critical gap: historical light client data was never generated in Lighthouse, and there is a single root cause for this. -**Recency guard in `import_block_update_metrics_and_events`:** +The function `import_block_update_metrics_and_events` is designed so that it only notifies the light client server for blocks that are within 32 slot-durations of the current time: ```rust // Do not trigger light_client server update producer for old blocks, to extra work @@ -44,35 +44,38 @@ if self.config.enable_light_client_server } ``` -This is an optimization. When Lighthouse is syncing old blocks, it skips notifying the light client server to avoid extra computation. The assumption was: "nobody needs light client data for old blocks." That assumption now stands as a barrier to what this project aims to achieve. - -**Storage gap in `get_light_client_bootstrap`:** - -```rust -// we currently have no backfill mechanism for these values. -// Therefore, sync_committee_branch and sync_committee are only persisted while a node is synced. -#[allow(clippy::type_complexity)] -pub fn get_light_client_bootstrap( - &self, - store: &BeaconStore, - block_root: &Hash256, - finalized_period: u64, - chain_spec: &ChainSpec, -) -> Result, ForkName)>, BeaconChainError> {...} +This code is an intentional optimization: it ensures Lighthouse avoids the extra computation of generating light client updates while racing to catch up during initial sync or backfill. This keeps the live sync path efficient by only processing recent blocks relevant to light client data consumers. + +However, this singular recency guard causes two key downstream effects: + +- **LightClientUpdate is never persisted for historical periods:** Because `recompute_and_cache_updates` is never invoked for old blocks, Lighthouse never generates or stores historical `LightClientUpdate` data. +- **`get_light_client_bootstrap` fails for historical roots:** As documented in the source: + + ```rust + // we currently have no backfill mechanism for these values. + // Therefore, sync_committee_branch and sync_committee are only persisted while a node is synced. + #[allow(clippy::type_complexity)] + pub fn get_light_client_bootstrap( + &self, + store: &BeaconStore, + block_root: &Hash256, + finalized_period: u64, + chain_spec: &ChainSpec, + ) -> Result, ForkName)>, BeaconChainError> {...} ``` + ``` + + The function tasked with serving `get_light_client_bootstrap` only persists `sync_committee_branch` and `sync_committee` for the periods being actively synced. If a user requests historical roots (pre-checkpoint or unbackfilled blocks), it fails, simply because the persistence never occurred for those historical periods. -Even if you synced old blocks, Lighthouse doesn't retroactively compute and store the `sync_committee_branch`. It only stores this for the blocks you processed while already synced. - -The combined effect: a Lighthouse node that synced from genesis or checkpoint cannot serve `LightClientBootstrap` or historical `LightClientUpdates` because it never generated or stored the proofs. - -Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspired by the LC sync strategy: +**Our solution does not touch this live import optimization.** +Instead, we add a dedicated post-sync background task that leverages the exact same data persistence pathway—calling `recompute_and_cache_updates`, which in turn writes to `store_light_client_update`, `store_sync_committee_branch`, and `store_current_sync_committee`. This new task operates over historical periods after live sync has completed, bypassing the import guard entirely. This means Lighthouse can safely and efficiently generate and persist all historical light client data needed, without risking performance of the main sync path, and without loosening the guard that protects it. 1. **Make Lighthouse "collect" historical light client data** — The consensus spec tests verify that a client can construct the full sequence of light client objects, but Lighthouse currently throws away historical data. We need to ensure Lighthouse can generate and persist, for every period it has state available: - `LightClientUpdate` for every sync committee period - `sync_committee_branch` for finalized checkpoint blocks - `LightClientBootstrap` data for historical finalized blocks -2. **Design the backfill API** — Once nodes have the data, we need a way to request it from peers for periods they don't have locally. This is a libp2p request/response protocol, following the pattern of Etan's `LightClientDataBackfillByRange` spec. Key design questions already resolved by the spec: +2. **Design the backfill API** — Once nodes have the data, we need a way to request it from peers for periods they don't have locally. This is a libp2p request/response protocol, following the pattern of `LightClientDataBackfillByRange`'s spec. Key design questions already resolved by the spec: - Transport: libp2p req/resp - Rate limiting: capped at 256 epochs per request - Proof-of-honesty: every field in the response is independently Merkle-verifiable by the requester, so a lying peer is detectable, not merely trusted @@ -85,16 +88,18 @@ Our solution is a trust-minimized checkpoint sync strategy borrowed from/inspire **The Problem:** Lighthouse already computes light client proofs for every block during sync, but the recency guard in `import_block_update_metrics_and_events` and the bounded channel (`LIGHT_CLIENT_SERVER_CHANNEL_CAPACITY = 32`) prevent historical data from reaching the database. Additionally, `get_light_client_bootstrap` explicitly lacks a backfill mechanism. The result: checkpoint-synced nodes cannot serve historical LC data to peers. -**The Solution (Local Collection — Phase 1a):** Implement a post-sync backfill task that walks the finalized chain from the node's **earliest available state** to the present. (Note: this is bounded by whichever `BeaconState`s the node actually retains — a checkpoint-synced, non-archive node cannot walk back to Altair; only an archive node can. Backfill coverage is therefore a function of node configuration, not a project guarantee of full Altair-to-present coverage on every node type.) For each sync committee period, identify the best block (highest sync aggregate participation), load its state from the freezer DB, and call `recompute_and_cache_updates` to store the canonical `LightClientUpdate` and `SyncCommitteeBranch`. +**The Solution** +**Phase 1a - Local Collection:** Implement a post-sync backfill task that walks the finalized chain backward, from the most recently finalized sync committee period to the node's earliest available *state* (per `store.get_historic_state_limits()`). (Note: this is bounded by whichever `BeaconState`s the node actually retains — a checkpoint-synced, non-archive node cannot walk back to Altair; only an archive node can. Backfill coverage is therefore a function of node configuration, not a project guarantee of full Altair-to-present coverage on every node type.) For each sync committee period, call the existing block-import light client update path — `recompute_and_cache_updates` — for every block in the period. It already retains only the spec-best update per period via `is_better_light_client_update`, so no separate candidate-selection step is needed; it stores the canonical `LightClientUpdate` and `SyncCommitteeBranch` for whichever block wins that comparison. This task: -- Spawns when `BackFillState::Completed` is set +- Spawns once the node reaches `SyncState::Synced`, gated on `store.get_historic_state_limits()` rather than `BackFillState::Completed` — block backfill only guarantees historical *blocks* are available, not the *states* this task needs, so it doesn't need to wait for it and can run concurrently with it +- Walks backward, from the most recently finalized sync committee period toward the node's earliest available state, so the periods most likely to be requested by peers are backfilled first - Is resumable (checks `store.get_light_client_update(period)` before processing) - Is pausable and low-priority (yields to validator duties) - Only works for periods where the node has the `BeaconState` -- Reuses Lighthouse's **existing** `DBColumn::LightClientUpdate` / `SyncCommitteeBranch` / `SyncCommittee` storage — this phase does not introduce a new DB column for `LightClientEpochData`. `LightClientEpochData` is a **wire/transport container** defined by Etan's spec for the p2p endpoint below; it is assembled on demand from the existing stored fields when serving a request, not persisted as its own schema. Etan's spec is explicitly marked as a draft ("TBD and likely not yet optimal"), so we avoid committing local storage to a container shape that may still change. +- Reuses Lighthouse's **existing** `DBColumn::LightClientUpdate` / `SyncCommitteeBranch` / `SyncCommittee` storage — this phase does not introduce a new DB column for `LightClientEpochData`. `LightClientEpochData` is a **wire/transport container** defined by the spec draft for the p2p endpoint below; we avoid committing local storage to a container shape that may still change. Note: this storage retains only the period's winning block, not per-epoch data for every slot — sufficient for Phase 1a's goal (`LightClientUpdate` history + bootstrap), but not by itself enough to serve the full `LightClientDataBackfillByRange` endpoint as specced (see Phase 1b scoping note below) -**The Solution (P2P Serving — Phase 1b):** Implement the `LightClientDataBackfillByRange` libp2p endpoint as specified in Etan's draft: +**Phase 1b - P2P Serving:** Implement the `LightClientDataBackfillByRange` libp2p endpoint as specified in spec draft: ``` /eth2/beacon_chain/req/light_client_data_backfill_by_range/0/ @@ -111,6 +116,12 @@ Implementation follows the pattern of the existing `handle_light_client_updates_ `LightClientEpochData` contains per-epoch raw block data including `sync_committee_bits`, `sync_aggregate_branch`, `finalized_checkpoint`, and `current_sync_committee` — everything a receiver needs to independently simulate `is_better_update` and verify the canonical best update for a period. +**Storage scoping decision:** Phase 1a's storage (`LightClientUpdate`/`SyncCommitteeBranch`/`SyncCommittee`) retains only the winning block per sync committee period — not per-slot data for every epoch in that period. The `LightClientDataBackfillByRange` endpoint as drafted requests by individual `epoch`, and each `LightClientEpochData` response is expected to contain full per-slot `block_data` for that epoch, including epochs that did not win their period's `is_better_light_client_update` comparison. This is a real granularity gap: the endpoint's purpose (per the spec: "verifying every single field... then simulating is_better_update") is to let the requester independently recompute which block was best, rather than trust the server's selection — which requires access to the non-winning epochs' raw data too. + +**For this project's MVP, we're building Option A: scope the endpoint to what Phase 1a already stores** — serve period-best `LightClientUpdate` history rather than the full per-epoch `LightClientEpochData` container. This requires no new storage, ships against the existing consensus-specs `light_client_data_collection` test format (which does not test `LightClientEpochData`), and fits the project timeline. The trade-off is accepted for MVP: a requester has to trust the server's "best block" selection rather than independently verify it from raw per-epoch data, so this does not yet deliver the endpoint's full trust-minimization goal as drafted. + +**Option B — full per-slot storage for every backfilled epoch, matching the spec as drafted — is the eventual path**, targeted as a stretch goal beyond this project's MVP scope (or a follow-on after EPF). It requires new storage design (roughly 32x more records than Option A) and should ideally mirror Nimbus's existing implementation (PR #8445) rather than Lighthouse inventing its own independently. + Key constraints: - Only serves finalized data - Fork context determined from last non-empty `block_data[i]` (or `epoch` if all empty) @@ -122,7 +133,7 @@ Key constraints: 1. **Unit tests** - `recompute_and_cache_updates` produces a correct `LightClientUpdate` and `SyncCommitteeBranch` for a known-good historical period - - Constructing `LightClientEpochData` from stored `LightClientUpdate`/`SyncCommitteeBranch` data round-trips correctly against Etan's container spec + - Constructing `LightClientEpochData` from stored `LightClientUpdate`/`SyncCommitteeBranch` data round-trips correctly against the container spec - Best-candidate selection (highest sync aggregate participation) picked correctly across a period with missed slots 2. **Integration tests** @@ -240,7 +251,7 @@ Phase 1 is independently useful on its own (it fixes a real gap in Lighthouse's - **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. - **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. - **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking. -- **No BeaconState modifications required:** Etan's revised spec avoids BeaconState modifications entirely (an improvement over the earlier EIP-7658 approach, which needed one). The trust model relies on Merkle-proved `LightClientEpochData` rather than enshrined state tracking, which is why this project does not require a hard fork. +- **No BeaconState modifications required:** The revised spec avoids BeaconState modifications entirely (an improvement over the earlier EIP-7658 approach, which needed one). The trust model relies on Merkle-proved `LightClientEpochData` rather than enshrined state tracking, which is why this project does not require a hard fork. ## Goal of the project From f7cde2f60c6719b6e43a1e251e827eef14ebd019 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Sun, 13 Sep 2026 14:51:23 +0100 Subject: [PATCH 7/8] edited proposal based on Etan's review comments --- ...ighthouse-decentralized-checkpoint-sync.md | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index ecfe3e9..aa59d99 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -89,15 +89,15 @@ Instead, we add a dedicated post-sync background task that leverages the exact s **The Problem:** Lighthouse already computes light client proofs for every block during sync, but the recency guard in `import_block_update_metrics_and_events` and the bounded channel (`LIGHT_CLIENT_SERVER_CHANNEL_CAPACITY = 32`) prevent historical data from reaching the database. Additionally, `get_light_client_bootstrap` explicitly lacks a backfill mechanism. The result: checkpoint-synced nodes cannot serve historical LC data to peers. **The Solution** -**Phase 1a - Local Collection:** Implement a post-sync backfill task that walks the finalized chain backward, from the most recently finalized sync committee period to the node's earliest available *state* (per `store.get_historic_state_limits()`). (Note: this is bounded by whichever `BeaconState`s the node actually retains — a checkpoint-synced, non-archive node cannot walk back to Altair; only an archive node can. Backfill coverage is therefore a function of node configuration, not a project guarantee of full Altair-to-present coverage on every node type.) For each sync committee period, call the existing block-import light client update path — `recompute_and_cache_updates` — for every block in the period. It already retains only the spec-best update per period via `is_better_light_client_update`, so no separate candidate-selection step is needed; it stores the canonical `LightClientUpdate` and `SyncCommitteeBranch` for whichever block wins that comparison. +**Phase 1a - Local Collection:** Implement a post-sync backfill task, gated behind an explicit opt-in flag (off by default, following the existing `--archive`/`--reconstruct-historic-states` precedent). The task walks the finalized chain forward, from the node's earliest available state (per `store.get_historic_state_limits()`) toward the present. (Note: coverage is bounded by whichever `BeaconState`s the node actually retains. A checkpoint-synced, non-archive node cannot walk back to Altair; only an archive node can.) For each sync committee period, call the existing block-import light client update path — `recompute_and_cache_updates` — for every block in the period, in slot order. It already retains only the spec-best update per period via `is_better_light_client_update`, so no separate candidate-selection step is needed. This task: -- Spawns once the node reaches `SyncState::Synced`, gated on `store.get_historic_state_limits()` rather than `BackFillState::Completed` — block backfill only guarantees historical *blocks* are available, not the *states* this task needs, so it doesn't need to wait for it and can run concurrently with it -- Walks backward, from the most recently finalized sync committee period toward the node's earliest available state, so the periods most likely to be requested by peers are backfilled first -- Is resumable (checks `store.get_light_client_update(period)` before processing) +- Runs only when explicitly enabled via an opt-in flag, off by default +- Walks forward, from the node's earliest available state toward the present, to work with (not against) the `historic_state_cache`'s incremental replay +- Is resumable via a dedicated marker, set only once every block in a period has been processed. - Is pausable and low-priority (yields to validator duties) - Only works for periods where the node has the `BeaconState` -- Reuses Lighthouse's **existing** `DBColumn::LightClientUpdate` / `SyncCommitteeBranch` / `SyncCommittee` storage — this phase does not introduce a new DB column for `LightClientEpochData`. `LightClientEpochData` is a **wire/transport container** defined by the spec draft for the p2p endpoint below; we avoid committing local storage to a container shape that may still change. Note: this storage retains only the period's winning block, not per-epoch data for every slot — sufficient for Phase 1a's goal (`LightClientUpdate` history + bootstrap), but not by itself enough to serve the full `LightClientDataBackfillByRange` endpoint as specced (see Phase 1b scoping note below) +- Reuses Lighthouse's **existing** `DBColumn::LightClientUpdate` / `SyncCommitteeBranch` / `SyncCommittee` storage — this phase does not introduce a new DB column for `LightClientEpochData`. `LightClientEpochData` is a **wire/transport container** defined by the spec for the p2p endpoint below; we avoid committing local storage to a container shape that's still evolving. Note: this storage retains only the period's winning block, not per-epoch data for every slot — sufficient for Phase 1a's goal (`LightClientUpdate` history + bootstrap), but not by itself enough to serve the full `LightClientDataBackfillByRange` endpoint as specced (see Phase 1b scoping note below) **Phase 1b - P2P Serving:** Implement the `LightClientDataBackfillByRange` libp2p endpoint as specified in spec draft: @@ -114,27 +114,27 @@ Implementation follows the pattern of the existing `handle_light_client_updates_ - Query stored `LightClientUpdate`/`SyncCommitteeBranch` data for the requested range and assemble each `LightClientEpochData` response entry - Stream responses back via `SendResponse` -`LightClientEpochData` contains per-epoch raw block data including `sync_committee_bits`, `sync_aggregate_branch`, `finalized_checkpoint`, and `current_sync_committee` — everything a receiver needs to independently simulate `is_better_update` and verify the canonical best update for a period. - -**Storage scoping decision:** Phase 1a's storage (`LightClientUpdate`/`SyncCommitteeBranch`/`SyncCommittee`) retains only the winning block per sync committee period — not per-slot data for every epoch in that period. The `LightClientDataBackfillByRange` endpoint as drafted requests by individual `epoch`, and each `LightClientEpochData` response is expected to contain full per-slot `block_data` for that epoch, including epochs that did not win their period's `is_better_light_client_update` comparison. This is a real granularity gap: the endpoint's purpose (per the spec: "verifying every single field... then simulating is_better_update") is to let the requester independently recompute which block was best, rather than trust the server's selection — which requires access to the non-winning epochs' raw data too. - -**For this project's MVP, we're building Option A: scope the endpoint to what Phase 1a already stores** — serve period-best `LightClientUpdate` history rather than the full per-epoch `LightClientEpochData` container. This requires no new storage, ships against the existing consensus-specs `light_client_data_collection` test format (which does not test `LightClientEpochData`), and fits the project timeline. The trade-off is accepted for MVP: a requester has to trust the server's "best block" selection rather than independently verify it from raw per-epoch data, so this does not yet deliver the endpoint's full trust-minimization goal as drafted. - -**Option B — full per-slot storage for every backfilled epoch, matching the spec as drafted — is the eventual path**, targeted as a stretch goal beyond this project's MVP scope (or a follow-on after EPF). It requires new storage design (roughly 32x more records than Option A) and should ideally mirror Nimbus's existing implementation (PR #8445) rather than Lighthouse inventing its own independently. +`LightClientEpochData` contains per-epoch raw block data (`sync_committee_bits`, `sync_aggregate_branch`, `finalized_checkpoint`) plus a nested `bootstrap_data: LightClientBootstrapData` container (`current_sync_committee`, `current_sync_committee_branch`, `execution_block_hash`, `execution_branch`) — everything a receiver needs to independently simulate `is_better_update` and verify the canonical best update for a period. Key constraints: - Only serves finalized data - Fork context determined from last non-empty `block_data[i]` (or `epoch` if all empty) -- Rate-limited to 256 epochs per request +- Response size capped at `MAX_REQUEST_LIGHT_CLIENT_EPOCH_DATA := 256` epochs per request. + +**Storage scoping decision:** Phase 1a's storage (`LightClientUpdate`/`SyncCommitteeBranch`/`SyncCommittee`) retains only the winning block per sync committee period — not per-slot data for every epoch in that period. The `LightClientDataBackfillByRange` endpoint as drafted requests by individual `epoch`, and each `LightClientEpochData` response is expected to contain full per-slot `block_data` for that epoch, including epochs that did not win their period's `is_better_light_client_update` comparison. This is a real granularity gap: the endpoint's purpose (per the spec: "verifying every single field... then simulating is_better_update") is to let the requester independently recompute which block was best, rather than trust the server's selection — which requires access to the non-winning epochs' raw data too. + +**MVP endpoint scope:** Phase 1a's storage (`LightClientUpdate`/`SyncCommitteeBranch`/`SyncCommittee`) retains only the winning block per sync committee period — not per-slot data for every epoch in that period. The `LightClientDataBackfillByRange` endpoint as specced requests by individual `epoch`, with each `LightClientEpochData` response expected to contain full per-slot `block_data`, including epochs that didn't win their period's `is_better_light_client_update` comparison — because the endpoint's purpose is letting the requester independently recompute which block was best, not trust the server's selection. + +**For this project's MVP, Phase 1b serves period-best `LightClientUpdate` history** rather than the full per-epoch `LightClientEpochData` container. This requires no new storage, ships against the existing consensus-specs `light_client_data_collection` test format, and fits the project timeline. The trade-off: a requester has to trust the server's "best block" selection rather than independently verify it from raw per-epoch data. -**On-Demand Fallback:** For periods not yet backfilled, modify `get_light_client_updates` and `get_light_client_bootstrap` to check the DB first, then fall back to computing from archived states if available (archive-mode nodes only). For pre-checkpoint data on a non-archive node, local generation is impossible — the node must request it from peers via the endpoint above. +**Future work (not being built in this project):** Full per-slot storage for every backfilled epoch, matching the endpoint as specced — roughly 256x more records than the MVP (1 per sync committee period → 1 per epoch, 256 epochs/period). Jeff is independently investigating this for Prysm; worth coordinating with him rather than designing it twice. This would also need to mirror whatever storage shape Nimbus settles on, given the spec is still evolving. **Testing (Phase 1a/1b):** 1. **Unit tests** - `recompute_and_cache_updates` produces a correct `LightClientUpdate` and `SyncCommitteeBranch` for a known-good historical period - Constructing `LightClientEpochData` from stored `LightClientUpdate`/`SyncCommitteeBranch` data round-trips correctly against the container spec - - Best-candidate selection (highest sync aggregate participation) picked correctly across a period with missed slots + - Best-candidate selection picked correctly across a period with missed slots, using the full `is_better_light_client_update` ranking. 2. **Integration tests** - Post-sync backfill completes for all periods from earliest available state to present @@ -246,8 +246,8 @@ Phase 1 is independently useful on its own (it fixes a real gap in Lighthouse's ## Possible challenges -- **Protocol evolution risk:** Etan's HackMD spec is explicitly marked as a draft ("TBD and likely not yet optimal"). The `LightClientEpochData` container, chunk sizes, and endpoint paths may change during implementation. We build defensively, pinning only the parts that are stable (the existing `LightClientUpdate` storage, the `is_better_update` ranking logic) and adapting the transport layer as the spec stabilizes. -- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. If a node pruned states before backfill completed, or if it checkpoint-synced and never had old states, it cannot generate historical light client data for those periods. We document that full historical coverage requires either archive node configuration or fetching missing data from peers (the backfill API) — it is not something the local backfill task alone can guarantee on every node type. +- **Protocol evolution risk:** Etan's HackMD spec is a draft and not yet optimal. The `LightClientEpochData` container, chunk sizes, and endpoint paths may change during implementation. We build defensively, pinning only the parts that are stable (the existing `LightClientUpdate` storage, the `is_better_update` ranking logic) and adapting the transport layer as the spec stabilizes. +- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. Beyond ordinary pruning, there is a real gap: block backfill covers months of history for which no corresponding `BeaconState` exists, and there is no libp2p protocol today to backfill historical states for that range. Local `LightClientUpdate` computation is impossible for that window regardless of node configuration — this isn't within Phase 1a's scope and is flagged as an open protocol question. - **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. - **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. - **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking. From 03d4dbce3a0dc9d708a160b6f8391c98769dbdb7 Mon Sep 17 00:00:00 2001 From: AbolareRoheemah Date: Sun, 13 Sep 2026 23:44:38 +0100 Subject: [PATCH 8/8] updated explanation on state availability for backfill --- projects/lighthouse-decentralized-checkpoint-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/lighthouse-decentralized-checkpoint-sync.md b/projects/lighthouse-decentralized-checkpoint-sync.md index aa59d99..4b74b5c 100644 --- a/projects/lighthouse-decentralized-checkpoint-sync.md +++ b/projects/lighthouse-decentralized-checkpoint-sync.md @@ -247,7 +247,7 @@ Phase 1 is independently useful on its own (it fixes a real gap in Lighthouse's ## Possible challenges - **Protocol evolution risk:** Etan's HackMD spec is a draft and not yet optimal. The `LightClientEpochData` container, chunk sizes, and endpoint paths may change during implementation. We build defensively, pinning only the parts that are stable (the existing `LightClientUpdate` storage, the `is_better_update` ranking logic) and adapting the transport layer as the spec stabilizes. -- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. Beyond ordinary pruning, there is a real gap: block backfill covers months of history for which no corresponding `BeaconState` exists, and there is no libp2p protocol today to backfill historical states for that range. Local `LightClientUpdate` computation is impossible for that window regardless of node configuration — this isn't within Phase 1a's scope and is flagged as an open protocol question. +- **State Availability for Backfill:** The backfill task requires loading historical `BeaconStates` from the freezer DB. Beyond ordinary pruning, coverage of the oldest history is a function of how many long-running, archive, or reconstructed nodes exist and continue to serve it. Hence, there's no protocol guarantee it survives forever, and this project builds the redistribution mechanism, not a guarantee of underlying availability. - **Performance During Sync:** While the backfill task runs in the background, loading and hashing old states is CPU and I/O intensive. We must ensure it yields to validator duties and does not starve the node of resources. The task should be pausable and low-priority. - **Channel Overflow:** If we remove the recency guard entirely instead of using post-sync backfill, the light client server channel (currently bounded to 32 slots) will overflow during fast sync, dropping events. The post-sync backfill approach avoids this, but we must verify that the `prev_block_cache` (also size 32) does not become a bottleneck if we repurpose the flow. - **Beacon Sync Chunk Proofs:** Designing Merkle multi-proofs for arbitrary fixed-size byte ranges of an SSZ container is non-trivial. The proofs must be efficient to generate (without rehashing the entire state) and compact enough to not negate the benefit of chunking.