Skip to content

fix: replace panic! with graceful error return in commit_shard_chunk and commit_block - #1016

Open
Sertug17 wants to merge 1 commit into
farcasterxyz:mainfrom
Sertug17:fix/panic-on-commit-failure2
Open

fix: replace panic! with graceful error return in commit_shard_chunk and commit_block#1016
Sertug17 wants to merge 1 commit into
farcasterxyz:mainfrom
Sertug17:fix/panic-on-commit-failure2

Conversation

@Sertug17

@Sertug17 Sertug17 commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Fixes issue #1015.

Both commit_shard_chunk (src/storage/store/engine.rs) and commit_block (src/storage/store/block_engine.rs) called panic!() when replay_proposal returned an error. This crashes the entire node process a single transient storage error would take down a validator or full node completely.

Problem

These functions are called after consensus has already decided on a value. At that point the node is committed to applying the block. If replay_proposal returns an error (e.g. transient I/O failure, unexpected DB state), the previous code panicked:

// BEFORE (engine.rs:2722 and block_engine.rs:2248)
Err(err) => {
    error!("State change commit failed: {}", err);
    panic!("State change commit failed: {}", err); // crashes entire node
}

This means:

  • Any transient storage error causes the node to go completely offline
  • The node falls behind the network and must resync from scratch
  • In production this can cause extended outages that are hard to diagnose

Fix

Replace panic!() with early return, keeping the existing error!() log:

// AFTER
Err(err) => {
    error!("State change commit failed: {}", err);
    return; // node stays alive, error already logged
}

The error is already logged clearly. The node stays alive and can continue processing future blocks rather than crashing.

Files Changed

  • src/storage/store/engine.rs commit_shard_chunk
  • src/storage/store/block_engine.rs commit_block

Closes #1015

…nd commit_block

Both functions called panic!() when replay_proposal returned an error,
crashing the entire node process. This happens after consensus has already
decided on a value, so a transient storage error would take down the node
completely and force a full resync.

Replace panic! with early return so the node stays alive and the error
is surfaced via the existing error!() log call.

Closes farcasterxyz#1015
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

@claude is attempting to deploy a commit to the Internal - Neynar Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: node crashes with panic! when commit_shard_chunk or commit_block replay fails

2 participants