headerfs: Implement Recovery on Read for Corrupted Headers

[mohamedawnallah]

Description

Currently, when a user experiences a partial write, the system fails on subsequent read operations of headers. This forces users to manually delete the corrupted headers to resolve the issue, which is not feasible for end users on mobile platforms.

We need to implement a solution that:

1. Ensures proper write operations to prevent header corruption (See headerfs: fail gracefully on header write #313)
2. Adds recovery capability when reading corrupted headers from partial writes

This will improve user experience by handling failure scenarios gracefully without requiring manual intervention, which is especially important for mobile users who lack access to header management tools.

The patch should detect corrupted headers during read operations and either repair them when possible or gracefully reset to a default state, eliminating the current hard failure that leaves the application in an unusable state.

Motivation and Context

Motivated by @Roasbeef's comment #313 (comment).

[mohamedawnallah]

Could you please @saubyk assign this to me. I would like to explore it later

[wolftivy]

I've been trying to use neutrino in production for a bitcoin app and this failure comes up often enough to make the library basically unusable without doing a periodic hard delete and re-download of the whole neutrino database. Is there a known workaround whereby I could detect the problem and programmatically delete only the last few headers or something?

[mohamedawnallah]

I've been trying to use neutrino in production for a bitcoin app and this failure comes up often

Thanks for reporting this. I remember when I opened this issue It was relevant in my mental picture. Can you provide the steps to reproduce this failure deterministically, including any specific environment parameters?

Is there a known workaround whereby I could detect the problem and programmatically delete only the last few headers or something?

Perhaps not yet. There may be a bit overlap with #317 and #324. With those PRs especially the latter, we validate the common headers and any error e.g corrupted headers in target neutrino headers store we propagate this and probably can be programmatically handled then