Skip to content

bug(hubble): Sync Failure in v1.18.0 Nodes ~1 Hour Post v1.19.0 Release (Root Cause Unclear) #2542

@thorwarrior

Description

@thorwarrior

What is the bug?
After the recent network upgrade of Farcaster Hubble nodes (observed via the latest GitHub release), legacy nodes running on older versions(v.1.18.0) can no longer synchronize data with the network. This manifests as 0 peers connected and stalled message sync progress. The lack of prior upgrade notifications or github release log has caused operational instability, leaving node operators unable to maintain network participation.

How can it be reproduced? (optional)
The issue was discovered approximately 1 hour after the release of v1.19.0, and the exact cause is currently unclear. Only can provide some informations here

Image

Image

Additional context (optional)

  1. Critical gaps in communication :

    • No upgrade announcements via official channels (e.g., docs, Warpcast channel, Twitter or ...).
    • The latest release notes lack backward compatibility warnings or migration steps(If has).
  2. Proposed solutions :

    • Publish upgrade notices with timelines and backward compatibility guarantees.
    • Document changes in release notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    s-triageNeeds to be reviewed, designed and prioritized

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions