-
-
Notifications
You must be signed in to change notification settings - Fork 428
Lodestar Weekly Standups 2026
The Lodestar team hosts planning/standup meetings weekly on Tuesdays at 3:00pm Universal Standard Time. These meetings allow the team to conduct release planning, prioritise tasks, sync on current issues, implementations and provide status updates on the Lodestar project. For more information or to participate in the public meetings, see https://github.com/ChainSafe/lodestar/discussions/categories/standups
Note that these notes are transcribed and summarized by AI language models and may not accurately reflect the context discussed.
Agenda: https://github.com/ChainSafe/lodestar/discussions/8699
Recording/Transcript: https://drive.google.com/drive/folders/1OlxCfZnQPyelKVETeBPeIzdEKBE1pOX6
Limited changes in current diff;
- Primarily Obol updates from Nico completed over holidays. No rush for immediate release; willing to wait for additional features nearing completion.
- RC target: January 16th (end of next week) if additional features ready. No devnet timeline yet for Glamsterdam implementations; specs still in flux.
Spec Status:
- Stake builders PR merged (24 hours before standup; no teams have implemented yet).
- Minor refactoring & fork choice PRs on consensus specs repo over holidays (loophole closure regarding proposals in EPBS reorg scenarios).
- Justin preparing new v1.7.0-alpha.0 spec release;
- No date revealed for devnet-0 and will anticipate more planning on this week's ACDC.
- Many refactoring PRs pending especially around partial withdrawal sweep and deposit request handling.
- NC not blocked on implementation; fork choice work proceeding. Tracking progress through Glamsterdam tracking epic issue: https://github.com/ChainSafe/lodestar/issues/8439
Implementation Status:
- No implementations started for stake builders (merged too recently); Prysm particularly blocked awaiting spec PR merge.
- EPBS Breakout room cancelled; focus shifted to ACD discussions Thursday.
- Infrastructure restructure playbook deployments completed with new hostname standardization.
- 20 Hoodie validators accidentally slashed during deployment (joke: testing slashing mechanism works).
- Feature groups (1-4) recovering; all operational within one day.
- Metrics now easier to read with clear differentiation between super-node and subscribe-all-subnet performance.
-
Feature 1 still problematic due to QUIC deployment crashes from segfaults discussed in late 2025.
- Matt and Cayman plan to capture a core dump this week to debug. If the issue lies within
napi-rs, the team may abandon that approach.
- Matt and Cayman plan to capture a core dump this week to debug. If the issue lies within
-
ZAPI Updates: Cayman completed work on "ZAPI" (formerly Napi-Zig) over the break. The library now exposes ~100% of the N-API surface.
- The current state is a low-level thin wrapper.
- Discussions are ongoing regarding the design of high-level interfaces to ensure memory safety and ease of use.
- Cayman suggested potentially scrapping the current high-level automatic wrapper design in favor of a better approach.
-
TreeView Integration:
- A significant decision point exists regarding integrating TreeView into
beacon-state. - Two different implementation approaches (one from Tuyen, one from Kai) need to be synthesized to resolve issues with synchronizing updates across child views.
-
Goal: The team aims to integrate TreeView into the
beacon-stateoptimistically by the end of January. This is a foundational step for both the state transition refactor and the bindings work. - It was clarified that work on bindings can start in parallel with TreeView integration, though final merging will depend on the Tree View implementation being correct.
- A significant decision point exists regarding integrating TreeView into
-
Low-Level vs. High-Level: Cayman confirmed that
Zappynow exposes 100% of the N-API surface as a thin, low-level wrapper. However, the team decided that the current implementation of the "automagic" high-level interface (which uses a hint system for type wrapping) is likely the wrong design direction. - Decision: The team agreed to likely scrap or deprecate the current high-level wrapper code in favor of a redesign. For the immediate future, developers should enforce using the low-level interface or work offline (specifically Nazar and Cayman) to design a better high-level abstraction layer that minimizes memory leak risks.
Dependency on Tree View Data Structures
- A major architectural decision was made regarding what data structure the bindings should actually wrap.
-
The Problem: Currently, the codebase operates on a
structversion of the beacon state (BeaconStateAllForks). However, the long-term goal is to use a "Union of Tree Views". - The Decision: The bindings integration should be built directly on top of the Tree View foundation rather than the current struct implementation. The goal is to have both the JavaScript bindings and the Zig state transition logic build from the same underlying data structure.
- Implication for Setters: This shift requires changing how state updates are handled. Instead of updating pointers to data (current struct approach), the bindings must use explicit function call setters, as the Tree View approach does not maintain direct pointers to the underlying data.
Parallel Execution & Timeline *. The team addressed whether the bindings work is strictly blocked by the unfinished Tree View implementation (specifically the synthesis of Kai and Tuyen's differing PRs).
- Non-Blocking Workflow: It was decided that work on bindings does not need to halt until the Tree View is perfectly merged. Developers can create the necessary interfaces and data structures now and start writing the binding logic against them.
-
Integration Goal: While the final "green checkmark" (passing CI/tests) will require the Tree View logic to be correct and merged, the team is targeting the end of January to have the Tree View integrated into the
beacon-state. This will serve as the stable foundation for the bindings to be fully merged.
Memory Safety Considerations Matthew raised concerns about memory leaks, specifically regarding handle scopes in the new bindings.
- Assessment: Cayman noted that because the current bindings are very thin, leaks are less likely to stem from the library itself and more likely to arise from how they are used in the implementation logic.
-
Plan: The team agreed to cross the bridge of strict memory leak profiling (e.g.,
escapable_handle_scope) later, as the immediate priority is stabilizing the interface design.
- The team is ready to switch from Yarn to PNPM.
- Nazar resolved a CI caching issue that was holding this back.
- The switch will require everyone to update their local workflows (e.g.,
pnpm installinstead ofyarn). - Infrastructure will need to merge their ready PR with new playbook commands for proper deployments.
- Nazar has prototype code for changes in the fork choice rule.
- He expects to have a PR open early next week.
Phil:
- Infrastructure playbooks deployed with new hostnames.
- Accidentally slashed 20 Hoodie validators (validates slashing works).
- Feature groups 1-2 operational; 3-4 recovering.
- Targeting v1.39 RC by January 16th.
- Planning Zig roadmapping call for January 15th to discuss bindings/integration design.
Robby:
- Scheduling 45-minute one-on-one sessions with each team member this week for onboarding.
Cayman: ZAPI Bindings:
- Fully exposed ZAPI library (100% of NAPI surface now exposed vs. prior 80%).
- Lowest-level interface complete (thin NAPI wrapper); higher-level "magical" bindings need additional work.
- Examples provided for async, thread-safe, class wrapping, normal functions.
- Memory leaks likely in usage patterns rather than bindings; will monitor during integration.
- Current higher-level wrapper approach may be wrong (hint system for type returns problematic); considering redesign; discussing offline with Nazar.
Lodestar-Z Work:
- ERA file support added: reader, writer, download script for mainnet data integration testing.
- Real data testing strategy: use ERA files to create state, feed blocks into state transition, capture metrics.
- Metrics testing: integrate with Bing's metrics PR; use perf/flame graphs on real data.
- SSZ core cleanup PR open (refactor SSZ tree API tweaks); part 2 anticipated.
- Config module cleanup PR (polishing).
Tree-View Design Discussion:
- Two competing approaches from Tuyen and Kai for fixing tree-view child synchronization (updates not syncing across child/parent views).
- Need to synthesize both approaches; requires deep review.
- Timeline estimate: ~2 weeks to resolve design.
- End-of-January goal: integrate tree view into state transition (create data structure interface).
- Beacon state union typing not hard-blocked by tree-view completion; can start interface definition independently.
Tuyen: Lodestar-Z State Transition:
- Changed integration approach: now creating interface + implementation simultaneously with local branch for incremental blocking identification.
-
Refactoring completed:
- Simplified
getBlockSignature - Moved API implementation to state transition per fork
- Stopped supporting bundling pre-Electra attestations (thinner test code) - not supporting old forks
- Simplified
-
Blockers identified:
- Remove shuffling cache from state transition (taken care of by Matt and Rahul)
- Stop calling specific API of beacon state allforks; only time we mutate is during state transition
- Will continue with the integration and try to target PR readiness before vacation
- Cannot provide exact timeline due to large codebase; proceeding incrementally.
- Additional refactor needed for pointer updates β explicit setter calls (tree-view incompatible).
- Tree-view Design: Local branch approach for integrating tree-view; can identify blockers during implementation.
Matt:
Cell-level DAG work in progress; acknowledges upcoming challenges.
- No real timeline on when it needs to be ready, but should be prioritized
QUIC debugging:
- Core dump capture planned this week (Node.js v24).
- If NAPI-RS bindings root cause, will abandon approach (long blocker).
- Next standup should have core dump for analysis.
Planning & Timeline:
- Updating waterfall/mermaid chart for January-February-March-April timeline.
- Wants confirmation by end-January tree-view integration realistic; if February, will adjust.
- Clarified: tree-view design review & merge not blocking state transition interface definition or binding work (can parallel-work with async review delays).
Kai: Tree-view API Implementation:
- Multiple PRs already merged for tree-view API implementations; can optimize/refactor.
- Main remaining work: tree-view data structure design (resolving Tuyen vs. Kai approaches).
- Post-API completion: integrate tree-view into beacon state.
- Goal end-of-January: integrate beacon state tree-view.
- Parallel work possible: implement current tree APIs while design reviews happen.
Bing:
Metrics PR:
- Ready for review; small usability issue resolved by forking library subset locally.
- Next phase: test metrics using ERA file support (from Cayman).
Epoch Processing:
- Ported final epoch process (rotating proposals, caching) to Zig.
- Various PR reviews completed.
Jeff:
- Working on blinded beacon block header tests.
- Currently debugging issues; expects resolution this week.
- Available for additional help. Will discuss internally with Cayman and Matthew to see what Jeff can tackle.
Rahul:
- Lodestar PR: deleting shuffling caching from epoch.
- Lodestar-Z PR: removing mutation transition code.
Vedant:
- Code cleanup focus: comments, logging fixes.
- E2E testing exploration: basic tests written and working.
- Backfill thread documentation: scenarios and edge cases outlined.
- Next steps: 7-10 days additional testing; call with Matt scheduled after.
Nazar:
PNPM Migration:
- Technically ready now (can switch immediately).
- Minor blocker: GitHub CI runner cache issue with node_modules (affecting CI time slightly).
- Debugging completed; will finalize by today.
- If only CI issue, proceeding anyway.
Fast Confirmation Rule:
- Prototype code complete for fork choice changes.
- PR ready by early next week.
Zappy/Zig Design:
- Discussing tree-view design approach with Cayman offline.
Yarn to PNPM Commands/Migration Notes:
- Commands identical (
yarn buildβpnpm build;yarn installβpnpm install). - Only difference:
pnpmrequires explicitpnpm install(yarn auto-installed). - Playbook PR ready for infrastructure updates.
- Corepack version pinning works; pnpm warns if package.json version older but still compatible.
- Robby: Schedule 45-min 1:1s with all team members this week (EST availability).
- Phil: Finalize Zig roadmapping call for January 15th.
- Matthew: Capture Quic core dump this week; schedule planning with Cayman on timeline.
- Cayman & Nazar: Discuss ZAPI higher-level interface design offline.
- Kai, Tuyen, Cayman: Resolve tree-view design over next 2 weeks; target Jan 31 for beacon state integration.
- Tuyen: Continue state transition interface definition & refactoring (parallel work possible).
- Bing: Prepare metrics testing with ERA file data.
- Nazar: Finalize PNPM migration (announce in chat if proceeding this week).
- Matt & Cayman: Prepare tree-view demo for January 15th call.
- Vedant: Complete 7-10 days testing; schedule call with Matt after.
- All: Expect PNPM command changes; watch chat for announcement.