feat!: implement fibre

[mcrakhman]

Fibre Protocol Flow

Fibre is a blob storage layer on top of Celestia. Instead of posting blobs directly on-chain (expensive), clients distribute blob shards across validators off-chain and only post a small on-chain transaction (MsgPayForFibre) as proof of storage.

Default Protocol Parameters

Parameter	Value
Original rows (K)	4096
Encoding ratio	0.25 (so total rows = 16384, parity rows = 12288)
Max validator count	100
Unique decoding security bits	100
Safety threshold	2/3 of stake
Liveness threshold	1/3 of stake
Max blob size	128 MiB
Min row size	64 bytes
Min rows per validator	148
Max rows per validator	4096
Validators needed for reconstruction	34
Upload/download concurrency	100

Upload Flow

1. Encode the blob

The raw data gets a 5-byte header (version + size), then is split into 4096 rows. Rows are extended using Reed-Solomon erasure coding — the 4096 original rows become 16384 total rows (12288 parity rows). A Merkle tree is built over all rows, producing a blob commitment (the Merkle root).

2. Create a Payment Promise

The client creates a PaymentPromise — a signed message saying "I promise to pay for storing this blob." It contains the chain ID, client's secp256k1 public key, namespace, blob size, commitment, blob version, height, and timestamp. The client signs this with its private key.

3. Assign shards to validators

Using a deterministic shuffle (seeded by the blob commitment), the 16384 rows are distributed across the active validator set (up to 100 validators). Each validator gets a subset of rows (their "shard") — at least 148 rows each, up to 4096 for the highest-staked validator. The shuffle uses Go-compatible ChaCha8 PRNG so both Rust client and Go server agree on the assignment.

4. Fan-out upload

For each validator (up to 100 concurrently), the client:

• Generates row inclusion proofs — each proof contains the row data plus a Merkle proof (sibling hashes from a depth-14 tree) proving the row belongs to the blob commitment
• Sends an UploadShard gRPC request with the payment promise, row proofs, and RLC coefficients
• Concurrency is bounded by the upload_concurrency semaphore (default: 100)

5. Collect signatures

Each validator verifies the shard (checks Merkle proofs, verifies the payment promise signature, validates RLC) and responds with an ed25519 validator signature over the payment promise. The client collects signatures until it has enough voting power (2/3+ of the validator set).

6. Build on-chain transaction

The collected signatures form a SignatureSet. The client builds MsgPayForFibre containing the payment promise and signature set, which gets submitted on-chain. The on-chain module verifies the signatures against the active validator set and processes payment.

Download Flow

1. Fetch validator set and assignments

The client fetches the validator set for the blob's height and recomputes the deterministic shard assignment (same ChaCha8 shuffle) to know which validators hold which rows.

2. Adaptive fan-out download

The client doesn't contact all validators at once. It starts with a subset and expands (up to 100 concurrent tasks):

• Requests DownloadShard from selected validators
• Each response contains row inclusion proofs for that validator's assigned rows
• As responses arrive, each row proof is verified against the blob commitment (Merkle proof check) and stored via set_row

3. Reconstruct

Once 4096 original rows (or any 4096 of the 16384 total rows) are collected, the blob can be reconstructed using Reed-Solomon decoding. The header is decoded from the first row to extract the original data. Only 34 validators (1/3 of 100) are needed for reconstruction.

Proofs Used

Proof	What it proves	Where used
Row Inclusion Proof (Merkle proof, depth 14)	A specific row belongs to the blob commitment	Upload (client->validator), Download (validator->client)
Payment Promise signature (secp256k1 ECDSA)	Client authorized payment for this blob	Upload (client signs), on-chain verification
Validator signature (ed25519)	Validator received and verified the shard	Upload (validator->client), on-chain in MsgPayForFibre
Blob commitment (Merkle root over 16384 rows)	Integrity of the entire blob	Everywhere — ties all proofs together
RLC verification (random linear combination, 4096 coefficients)	Validators actually store the data, not just the commitment	Upload server-side check

Server Side (Go)

UploadShard handler: Verifies the payment promise signature, checks Merkle proofs for each row, performs RLC verification, stores the shard in a local DB, and returns an ed25519 signature.

DownloadShard handler: Looks up the stored shard by blob ID, returns the row inclusion proofs to the requesting client.

On-chain (x/fibre module): MsgPayForFibre handler verifies the signature set has sufficient voting power (2/3+), checks the payment promise, and processes the storage payment.

[rach-id]

As per sync discussion: we will have an e2e test that submits a fibre blob and downloads it to make sure the flow works as expected.

Added a few comments from asking claude to review the PR.

[rach-id]

probably this file shouldn't be pushed

[rach-id]

Claude:

2. Upload upload_shards silently succeeds without enough signatures

  File: fibre/src/client/upload.rs:1530-1533

      // After the loop exits:
      Ok(())

  When the upload_shards loop exhausts all validators without the signature threshold being met, it returns Ok(()). The caller then calls sig_set.signatures() which will return Err(NotEnoughSignatures), so it
  does ultimately fail — but this is a confusing control flow. The upload_shards function's contract should be clearer about whether it's responsible for checking the threshold. More importantly, if
  sig_set.signatures() is the only guard, then there's a subtle race: if upload_shards returns Ok(()) after all tasks complete with errors (so sig_set has zero signatures), the error message will say "not
  enough voting power: collected 0, required X" — which is correct but not as helpful as "all validator uploads failed."

[rach-id]

Claude:

 8. Download loop doesn't cancel inflight tasks when original_rows is reached

  File: fibre/src/client/download.rs:626-628

  if unique_rows >= original_rows {
      break;
  }

  When enough rows are collected, the loop breaks, but already-spawned download tasks continue running in the background. Unlike uploads (where background delivery is intentional), downloads have no benefit
  from continuing — the blob is already reconstructable. These tasks consume bandwidth and connection slots unnecessarily.