Skip to content

feat(builder): observe beacon node blocks - #9931

Open
krisoshea-eth wants to merge 20 commits into
ChainSafe:unstablefrom
krisoshea-eth:krisoshea/lod-12-api-02-consume-bn-block-events-and-retrieve-fork-correct
Open

feat(builder): observe beacon node blocks#9931
krisoshea-eth wants to merge 20 commits into
ChainSafe:unstablefrom
krisoshea-eth:krisoshea/lod-12-api-02-consume-bn-block-events-and-retrieve-fork-correct

Conversation

@krisoshea-eth

@krisoshea-eth krisoshea-eth commented Aug 27, 2026

Copy link
Copy Markdown

Motivation

The Lodestar Builder needs a source beacon node API path for learning whether its bid was selected without joining libp2p. This PR implements the standard block SSE topic plus getBlockV2 as a bounded compatibility path.

Beacon APIs #630 and merged Lodestar #9832 also forward the signed winning block directly to the selected external Builder. The main purpose of that flow is to let the Builder help disseminate the block, with timely win notification as an additional benefit. Maintainer input is requested on whether this observer should remain as a compatibility fallback alongside direct delivery.

Changes

  • Subscribe only to the standard block SSE topic.
  • Deduplicate roots before asynchronous work and retain a FIFO window of 256 roots.
  • Skip locally pre-Gloas slots and retrieve each new post-Gloas block by root with getBlockV2.
  • Use the Builder abort signal for both the SSE stream and block requests.
  • Retry 404, server, timeout, and non-input transport failures for up to six attempts with five 200 ms delays. Other 4xx responses, cancellation, decoding failures, input fetch errors, and structural failures are terminal.
  • Treat response version metadata as fork authority, require a Gloas-compatible body, and verify the returned slot against the event.
  • Preserve the exact signed bid, including later-fork fields, exact-width values, and BUILDER_INDEX_SELF_BUILD.
  • Dispatch observations concurrently through isolated runOnBlock callbacks.

The existing API-client request timeout is unchanged. This PR bounds attempts and explicit retry delays, not total wall-clock or slot-relative time. Selection and reveal code will own deadline policy.

The observer starts after the existing genesis, configuration, readiness, Gloas, and Builder identity gates. It joins the clock and BuilderStatusTracker as a long-lived Builder duty and shares their abort controller.

This PR does not add p2p, block_gossip, canonical-chain filtering, local-bid matching, reveal behavior, metrics, reconnect, replay, restart recovery, multi-BN failover, or a new API endpoint.

The observer intentionally evaluates blocks before SELECT-01 registers its first production consumer so this compatibility path remains active and evidenced. Event-time executionOptimistic comes from the triggering event, while the response metadata is authoritative for the fork. Terminally failed roots remain consumed until FIFO eviction; REL-01 owns controlled reconciliation. Aggregate retrieval concurrency, observer-specific metrics, and block-root recomputation before financial decisions are tracked in SEC-01, QA-01, and SELECT-01 respectively.

API behavior and compatibility

The standard block event contains the slot, beacon block root, and execution optimism. getBlockV2 supplies the signed fork-correct block and Eth-Consensus-Version metadata. Imported non-head blocks remain valid observations, so head and head_v2 are not substitutes.

Lodestar emits block after state transition and fork-choice import. Root lookup checks fork choice for presence, then serves the block from the seen-block input cache or database. This ordering provides no expected Lodestar event-before-block window, but the Beacon API does not require equivalent ordering across clients, so bounded 404 retry remains a cross-client precaution.

The implementation audit is recorded in merged Builder docs PR #13. The current direct-Engine planning reconciliation is in draft Builder docs PR #18, and the reproducible real-BN and shutdown evidence is in draft Builder docs PR #19. Implementation evidence was posted to beacon-APIs #599. Marco's open Lodestar PoCs #9854, #9875, #9876, and #9896 explore optional event improvements separately.

The API-02 diff is limited to five Builder files and is based on current unstable, including merged #9832. The combined code passes the Builder type-check and focused observer tests.

Current unstable base: 1e9a530f98.

Specification baseline: consensus-specs v1.7.0-alpha.14.

Project issue: krisoshea-eth/lodestar#12.

Testing

Validated with Node 24.13.0 and pnpm 11.0.0:

  • 26 focused observer tests plus one Builder lifecycle wiring test;
  • 54 Builder unit tests;
  • Builder package type-check;
  • Builder package lint;
  • Builder package build and import check; and
  • git diff --check.

Coverage includes SSE wiring, cancellation, startup and shutdown wiring, Gloas and Heze output, exact signed-bid identity, slot consistency, duplicate suppression, bounded retry, decoding and structural failures, FIFO eviction, self-builds, stream failures, and callback isolation.

AI assistance disclosure

  • External Contributors: I have read the contributor guidelines and disclosed my usage of AI below.

AI assistance was used for codebase research, implementation, testing, and review. I reviewed and revised the submitted code and PR text, made manual edits and technical decisions, and take responsibility for the final result.

krisoshea-eth and others added 16 commits August 10, 2026 21:25
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct

# Conflicts:
#	packages/builder/src/builder.ts
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct

# Conflicts:
#	packages/builder/src/builder.ts
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct
…2-api-02-consume-bn-block-events-and-retrieve-fork-correct
@krisoshea-eth
krisoshea-eth marked this pull request as ready for review August 27, 2026 15:48
@krisoshea-eth
krisoshea-eth requested a review from a team as a code owner August 27, 2026 15:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03d4ebd1b4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/builder/src/services/blockObserver.ts Outdated
Comment thread packages/builder/test/unit/services/blockObserver.test.ts Outdated
Comment thread packages/builder/src/services/blockObserver.ts Outdated
@markolazic01

Copy link
Copy Markdown
Contributor

checked 1d23805, new changes look good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants