From ab861a1efbb2e458fc69b9ad787e1dc393c61b28 Mon Sep 17 00:00:00 2001 From: hanabi1224 Date: Wed, 25 Jun 2025 07:46:43 +0800 Subject: [PATCH 1/7] frc: f3-augmented-snapshot --- FRCs/frc-f3-augmented-snapshot.md | 138 ++++++++++++++++++++++++++++++ README.md | 1 + 2 files changed, 139 insertions(+) create mode 100644 FRCs/frc-f3-augmented-snapshot.md diff --git a/FRCs/frc-f3-augmented-snapshot.md b/FRCs/frc-f3-augmented-snapshot.md new file mode 100644 index 000000000..5c29ebc22 --- /dev/null +++ b/FRCs/frc-f3-augmented-snapshot.md @@ -0,0 +1,138 @@ +--- +fip: "####" +title: "Augment Filecoin Snapshot with F3 data" +author: "Hailong Mu (@hanabi1224)" +discussions-to: https://github.com/filecoin-project/go-f3/issues/480 +status: Draft +type: "FRC" +created: 2025-06-25 +--- + +# FRC-####: Augment Filecoin Snapshot with F3 data + +## Simple Summary + + +Downloading F3 finality certificates from scratch takes a long time and increases the p2p network bandwidth usage. +An F3 snapshot is proposed to be included in the Filecoin CAR snapshot to reduce F3 catchup time and p2p network bandwidth usage on bootstrapping +a Filecoin node with a Filecoin snapshot. + +## Abstract + + +We propose extending the Filecoin CAR snapshot with an F3 snapshot as a raw data block, and changing CAR roots to be a CID that points to a CBOR-encoded Filecoin snapshot header struct. + +## Motivation + + +The time cost and the network bandwidth usage for a new Filecoin node to catch up with all F3 finality certificates grow over time, which delays the readiness of the F3-aware V2 RPC APIs. By embedding an F3 snapshot into the current Filecoin CAR snapshot, both can be vastly reduced at the cost of a slightly increased Filecoin CAR snapshot size. + +## Specification + + +We propose the blow changes to the Filecoin CAR snapshot format. + +- Change CAR roots to be a CID that points to a CBOR-encoded [`SnapshotMetadata`](#snapshotmetadata) struct that is stored as the first data block in the CAR. +- Store the raw [`F3Snapshot`](#f3snapshot) bytes as the second data block in the CAR when `F3Data != nil` in the metadata. + +### SnapshotMetadata + +```go +type SnapshotMetadata { + HeadTipsetKey []Cid // required + F3Data *Cid // optional +} +``` + +### F3Snapshot + +An F3 snapshot contains one header block and N(N>0) data blocks in the below format: + +`[Header block] [Data block] [Data block] [Data block] ...` + +A header block is a CBOR-encoded [`F3SnapshotHeader`](#f3snapshotheader) with a length prefix in the below format: + +`[varint-encoded length] [CBOR-encoded F3SnapshotHeader]` + +### F3SnapshotHeader + +```go +type SnapshotHeader struct { + Version uint64 + FirstInstance uint64 + LatestInstance uint64 + InitialPowerTable gpbft.PowerEntries +} +``` + +A data block is a CBOR-encoded [`FinalityCertificate`](#finalitycertificate) with a length prefix in the below format: + +`[varint-encoded length] [CBOR-encoded FinalityCertificate]` + +### FinalityCertificate + +```go +// FinalityCertificate represents a single finalized GPBFT instance. +type FinalityCertificate struct { + // The GPBFT instance to which this finality certificate corresponds. + GPBFTInstance uint64 + // The ECChain finalized during this instance, starting with the last tipset finalized in + // the previous instance. + ECChain *gpbft.ECChain + // Additional data signed by the participants in this instance. Currently used to certify + // the power table used in the next instance. + SupplementalData gpbft.SupplementalData + // Indexes in the base power table of the certifiers (bitset) + Signers bitfield.BitField + // Aggregated signature of the certifiers + Signature []byte + // Changes between the power table used to validate this finality certificate and the power + // used to validate the next finality certificate. Sorted by ParticipantID, ascending. + PowerTableDelta PowerTableDiff `json:"PowerTableDelta,omitempty"` +} +``` + +Notes: +- `FinalityCertificate`s should be ordered by `GPBFTInstance` in ascending order, thus they can be validated and intermediate power tables can be generated while data blocks are being read in a stream. +- The first and last `FinalityCertificate` instances should match those in the header, respectively. + +## Backwards Compatibility + + +- A filecoin node should try to read a snapshot CAR in the proposed format, and fallback to the old format to maintain backward compatibility. +- CLI options should remain unchanged to make it transparent to the node users. +- The code change in all Filecoin nodes should be shipped with a network upgrade, and the Filecoin snapshot providers should only start publishing with the new format after the mainnet upgrade finishes to avoid potential errors during snapshot import for node users. + +## Test Cases + + +## Security Considerations + + +This change has minimal security implications as the additional F3 data are also stored in the node database, unencrypted. Key considerations: + +- **Integrity**: The F3 snapshot can be validated. +- **Performance** The F3 snapshot data blocks can be read, validated and imported in a stream. + +The change does not introduce new attack vectors or modify existing security properties of the protocol. + +## Incentive Considerations + + +Node users should experience faster F3 bootstrapping time and less network bandwidth usage. + +## Product Considerations + + +Nodes starting from a snapshot should not rely on the certificate exchange protocol to catch up with the F3 data because we expect this will get slower over time. A slow F3 catchup time leads to, e.g. + +- delay in the readiness of F3-aware RPC APIs + +## Implementation + + +## Future Work + + +## Copyright +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/README.md b/README.md index 9705c00a5..29147acb5 100644 --- a/README.md +++ b/README.md @@ -141,3 +141,4 @@ This improvement protocol helps achieve that objective for all members of the Fi | [0103](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0103.md) | Removal of the ExtendSectorExpiration method from the miner actor | FIP | Rod Vagg (@rvagg) | Draft | | [0106](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0106.md) | Removal of the ProveReplicaUpdates method from the miner actor | FIP | Rod Vagg (@rvagg) | Draft | | [0107](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0107.md) | Implicit Messages in Block Receipts | FIP | Rod Vagg (@rvagg) | Draft | +| [XXXX](https://github.com/filecoin-project/FIPs/blob/master/FRPCs/frc-f3-augmented-snapshot.md) | Implicit Messages in Block Receipts | FRC | Hailong Mu (@hanabi1224) | Draft | From 02335e966d1abe22002bf67863487bbcecbab5aa Mon Sep 17 00:00:00 2001 From: hanabi1224 Date: Mon, 30 Jun 2025 21:34:31 +0800 Subject: [PATCH 2/7] Apply suggestions from code review Co-authored-by: Steve Loeppky Co-authored-by: Rod Vagg --- FRCs/frc-f3-augmented-snapshot.md | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/FRCs/frc-f3-augmented-snapshot.md b/FRCs/frc-f3-augmented-snapshot.md index 5c29ebc22..16ab693a7 100644 --- a/FRCs/frc-f3-augmented-snapshot.md +++ b/FRCs/frc-f3-augmented-snapshot.md @@ -8,7 +8,7 @@ type: "FRC" created: 2025-06-25 --- -# FRC-####: Augment Filecoin Snapshot with F3 data +# FRC-####: Filecoin Snapshot Format ## Simple Summary @@ -20,7 +20,11 @@ a Filecoin node with a Filecoin snapshot. ## Abstract -We propose extending the Filecoin CAR snapshot with an F3 snapshot as a raw data block, and changing CAR roots to be a CID that points to a CBOR-encoded Filecoin snapshot header struct. +The Filecoin ecosystem existed for years without specifying the snapshot format. That was fine until the advent of F3 and the resulting need to update the format in a coordinated way. + +This document outlines: +- "v1": the original accepted format found implementations through 2025 and +- "v2": the extension to v1 with an F3 snapshot as a raw data block, and changing CAR roots to be a CID that points to a CBOR-encoded Filecoin snapshot header struct. ## Motivation @@ -30,9 +34,9 @@ The time cost and the network bandwidth usage for a new Filecoin node to catch u ## Specification -We propose the blow changes to the Filecoin CAR snapshot format. +We propose the below changes to the Filecoin CAR snapshot format. -- Change CAR roots to be a CID that points to a CBOR-encoded [`SnapshotMetadata`](#snapshotmetadata) struct that is stored as the first data block in the CAR. +- Change CAR root to be a CID that points to a CBOR-encoded [`SnapshotMetadata`](#snapshotmetadata) struct that is stored as the first data block in the CAR. - Store the raw [`F3Snapshot`](#f3snapshot) bytes as the second data block in the CAR when `F3Data != nil` in the metadata. ### SnapshotMetadata @@ -46,21 +50,21 @@ type SnapshotMetadata { ### F3Snapshot -An F3 snapshot contains one header block and N(N>0) data blocks in the below format: +An F3 snapshot contains one header block and N data blocks (where N>0) in the below format: `[Header block] [Data block] [Data block] [Data block] ...` A header block is a CBOR-encoded [`F3SnapshotHeader`](#f3snapshotheader) with a length prefix in the below format: -`[varint-encoded length] [CBOR-encoded F3SnapshotHeader]` +`[varint-encoded byte length of "CBOR-encoded F3SnapshotHeader"] [CBOR-encoded F3SnapshotHeader]` ### F3SnapshotHeader ```go -type SnapshotHeader struct { +type F3SnapshotHeader struct { Version uint64 - FirstInstance uint64 - LatestInstance uint64 + FirstInstance uint64 // The first FinalityCertificate.GPBFTInstance in the "data blocks" that follow the header. + LatestInstance uint64 // The last FinalityCertificate.GPBFTInstance in the "data blocks" that follow the header. InitialPowerTable gpbft.PowerEntries } ``` @@ -99,8 +103,8 @@ Notes: ## Backwards Compatibility -- A filecoin node should try to read a snapshot CAR in the proposed format, and fallback to the old format to maintain backward compatibility. -- CLI options should remain unchanged to make it transparent to the node users. +- A Filecoin node should try to read a snapshot CAR in the proposed format. A failure to successfully decode the blocked referenced as as the CAR's single root using the schema presented above, a snapshot reader may fallback to the old format and maintain backward compatibility. Additional failures to decode original snapshot format would indicate a fatal error. +- CLI options for implementations like Forest and Lotus should remain unchanged to make it transparent to the node users. - The code change in all Filecoin nodes should be shipped with a network upgrade, and the Filecoin snapshot providers should only start publishing with the new format after the mainnet upgrade finishes to avoid potential errors during snapshot import for node users. ## Test Cases @@ -131,6 +135,8 @@ Nodes starting from a snapshot should not rely on the certificate exchange proto ## Implementation +Lotus: https://github.com/filecoin-project/lotus/issues/13129 +Forest: ## Future Work From 78614aed173a45fc2ed25b82c80df400e379139a Mon Sep 17 00:00:00 2001 From: hanabi1224 Date: Tue, 1 Jul 2025 21:19:19 +0800 Subject: [PATCH 3/7] address comments --- FRCs/frc-0108.md | 136 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 118 insertions(+), 18 deletions(-) diff --git a/FRCs/frc-0108.md b/FRCs/frc-0108.md index 3da1298f5..c31ae653a 100644 --- a/FRCs/frc-0108.md +++ b/FRCs/frc-0108.md @@ -20,7 +20,7 @@ a Filecoin node with a Filecoin snapshot. ## Abstract -The Filecoin ecosystem existed for years without specifying the snapshot format. That was fine until the advent of F3 and the resulting need to update the format in a coordinated way. +The Filecoin ecosystem existed for years without specifying the snapshot format. That was fine until the advent of F3 and the resulting need to update the format in a coordinated way. This document outlines: - "v1": the original accepted format found implementations through 2025 and @@ -37,20 +37,21 @@ The time cost and the network bandwidth usage for a new Filecoin node to catch u We propose the below changes to the Filecoin CAR snapshot format. - Change CAR root to be a CID that points to a CBOR-encoded [`SnapshotMetadata`](#snapshotmetadata) struct that is stored as the first data block in the CAR. -- Store the raw [`F3Snapshot`](#f3snapshot) bytes as the second data block in the CAR when `F3Data != nil` in the metadata. +- Store the raw [`F3Data`](#f3data) bytes as the second data block in the CAR when `F3Data != nil` in the metadata. ### SnapshotMetadata ```go type SnapshotMetadata { + Version uint64 // required, format version for SnapshotMetadata. Only "2" is supported since "v1" was implied in the original format that predates `SnapshotMetadata`. HeadTipsetKey []Cid // required - F3Data *Cid // optional + F3Data *Cid // optional, points to F3Data structure. The only supported codec is "RAW" (0x55). } ``` -### F3Snapshot +### F3Data -An F3 snapshot contains one header block and N data blocks (where N>0) in the below format: +An F3 snapshot contains one header block and N data blocks (where N>0) in the below [CARv1](https://ipld.io/specs/transport/car/carv1)-like format: `[Header block] [Data block] [Data block] [Data block] ...` @@ -58,24 +59,31 @@ A header block is a CBOR-encoded [`F3SnapshotHeader`](#f3snapshotheader) with a `[varint-encoded byte length of "CBOR-encoded F3SnapshotHeader"] [CBOR-encoded F3SnapshotHeader]` +A data block is a CBOR-encoded [`FinalityCertificate`](#finalitycertificate) with a length prefix in the below format: + +`[varint-encoded byte length] [CBOR-encoded FinalityCertificate]` + +Notes: +- `FinalityCertificate`s should be ordered by `GPBFTInstance` in ascending order for sequential validation and intermediate power table calculation. This also ensures deterministic generation of F3 snapshot from a given F3 finality certificate chain. +- The first and last `FinalityCertificate` instances should match those in the header, respectively. +- This [CARv1](https://ipld.io/specs/transport/car/carv1)-like format is ideal for dumping blocks via streaming reads as the Header can be loaded first and minimal state is required for ongoing parsing. + ### F3SnapshotHeader ```go type F3SnapshotHeader struct { Version uint64 - FirstInstance uint64 // The first FinalityCertificate.GPBFTInstance in the "data blocks" that follow the header. - LatestInstance uint64 // The last FinalityCertificate.GPBFTInstance in the "data blocks" that follow the header. + FirstInstance uint64 + LatestInstance uint64 InitialPowerTable gpbft.PowerEntries } ``` -A data block is a CBOR-encoded [`FinalityCertificate`](#finalitycertificate) with a length prefix in the below format: - -`[varint-encoded length] [CBOR-encoded FinalityCertificate]` - ### FinalityCertificate ```go +// Defined at +// // FinalityCertificate represents a single finalized GPBFT instance. type FinalityCertificate struct { // The GPBFT instance to which this finality certificate corresponds. @@ -94,18 +102,109 @@ type FinalityCertificate struct { // used to validate the next finality certificate. Sorted by ParticipantID, ascending. PowerTableDelta PowerTableDiff `json:"PowerTableDelta,omitempty"` } + +// Defined at +// +// A chain of tipsets comprising a base (the last finalised tipset from which the chain extends). +// and (possibly empty) suffix. +// Tipsets are assumed to be built contiguously on each other, +// though epochs may be missing due to null rounds. +// The zero value is not a valid chain, and represents a "bottom" value +// when used in a Granite message. +type ECChain struct { + TipSets []*TipSet + + key ECChainKey `cborgen:"ignore"` + keyLazyLoader sync.Once `cborgen:"ignore"` +} + +// Defined at +type SupplementalData struct { + // Commitments is the Merkle-tree of instance-specific commitments. Currently + // empty but this will eventually include things like snark-friendly power-table + // commitments. + Commitments [32]byte `cborgen:"maxlen=32"` + // PowerTable is the DagCBOR-blake2b256 CID of the power table used to validate + // the next instance, taking lookback into account. + PowerTable cid.Cid // []PowerEntry +} + +// Defined at +type PowerTableDiff []PowerTableDelta + +// Defined at +// +// PowerTableDelta represents a single power table change between GPBFT instances. If the resulting +// power is 0 after applying the delta, the participant is removed from the power table. +type PowerTableDelta struct { + // Participant with changed power + ParticipantID gpbft.ActorID + // Change in power from base (signed). + PowerDelta gpbft.StoragePower + // New signing key if relevant (else empty) + SigningKey gpbft.PubKey `cborgen:"maxlen=48"` +} + +// Defined at +type ActorID uint64 + +// Defined at +type StoragePower = big.Int + +// Defined at +type PubKey []byte + +// Defined at +// +// TipSet represents a single EC tipset. +type TipSet struct { + // The EC epoch (strictly increasing). + Epoch int64 + // The tipset's key (canonically ordered concatenated block-header CIDs). + Key TipSetKey `cborgen:"maxlen=760"` // 20 * 38B + // Blake2b256-32 CID of the CBOR-encoded power table. + PowerTable cid.Cid + // Keccak256 root hash of the commitments merkle tree. + Commitments [32]byte `cborgen:"maxlen=32"` +} + +// Defined at +// +// TipSetKey is the canonically ordered concatenation of the block CIDs in a tipset. +type TipSetKey = []byte ``` -Notes: -- `FinalityCertificate`s should be ordered by `GPBFTInstance` in ascending order, thus they can be validated and intermediate power tables can be generated while data blocks are being read in a stream. -- The first and last `FinalityCertificate` instances should match those in the header, respectively. +### Filecoin snapshot v1 format + +We define the existing Filecoin snapshot format as v1 here for future reference. + +Filecoin snapshot v1 is in [CARv1](https://ipld.io/specs/transport/car/carv1/) format. + +The roots array in the `CarHeader` stores the tipset keys of the chain head in the snapshot. + +The data blocks are chain IPLD blocks generated in a deterministic depth-first traversal order during chain export. Thus a snapshot is deterministic for a given chain head and the number of state trees to include. (The details of the chain traversal algorithm can be found in Filecoin node implementations) + +```go +type CarHeader struct { + version Int + roots [&Any] +} +``` ## Backwards Compatibility -- A Filecoin node should try to read a snapshot CAR in the proposed format. A failure to successfully decode the blocked referenced as as the CAR's single root using the schema presented above, a snapshot reader may fallback to the old format and maintain backward compatibility. Additional failures to decode original snapshot format would indicate a fatal error. -- CLI options for implementations like Forest and Lotus should remain unchanged to make it transparent to the node users. -- The code change in all Filecoin nodes should be shipped with a network upgrade, and the Filecoin snapshot providers should only start publishing with the new format after the mainnet upgrade finishes to avoid potential errors during snapshot import for node users. +- A Filecoin node should try to read a snapshot CAR in the proposed format. A failure to successfully decode the block referenced as the CAR's single root using the schema presented above, a snapshot reader may fallback to the old format and maintain backward compatibility. Additional failures to decode original snapshot format would indicate a fatal error. +- CLI options for implementations like Forest, Lotus and Venus should remain unchanged to make it transparent to the node users. +- The code change in all Filecoin nodes should be shipped with a network upgrade, and the Filecoin snapshot providers should only start publishing with the new format after the mainnet upgrade finishes to avoid potential errors during snapshot import for node users. That is to say: + - Before NV27, + - a node reads both v1 and v2 snapshots + - a node generates v1 snapshot by default + - node provides publish and host v1 snapshots. + - After NV27, + - a node reads both v1 and v2 snapshots + - a node generates v2 snapshot by default + - node providers publish and host v2 snapshots. ## Test Cases @@ -116,7 +215,8 @@ Notes: This change has minimal security implications as the additional F3 data are also stored in the node database, unencrypted. Key considerations: - **Integrity**: The F3 snapshot can be validated. -- **Performance** The F3 snapshot data blocks can be read, validated and imported in a stream. +- **Performance** The F3 snapshot data blocks can be read, validated and imported in a stream, without requiring to hold the entire finality certificate chain in the RAM. To facilitate this, it might require some new API(s) in the CAR reader package. +- **Cyclic structure** Not applicable. The F3 snapshot does not build any cyclic graph during import and export, only a single block(certificate) is required to be held in the RAM. The change does not introduce new attack vectors or modify existing security properties of the protocol. From 7dac2e23cfa546a4a11975ebe9aefe9a780cdf4d Mon Sep 17 00:00:00 2001 From: Steve Loeppky Date: Tue, 1 Jul 2025 20:51:24 -0700 Subject: [PATCH 4/7] Minor readability cleanup by @biglep --- FRCs/frc-0108.md | 67 ++++++++++++++++++++++++------------------------ 1 file changed, 34 insertions(+), 33 deletions(-) diff --git a/FRCs/frc-0108.md b/FRCs/frc-0108.md index c31ae653a..9204caf58 100644 --- a/FRCs/frc-0108.md +++ b/FRCs/frc-0108.md @@ -31,10 +31,27 @@ This document outlines: The time cost and the network bandwidth usage for a new Filecoin node to catch up with all F3 finality certificates grow over time, which delays the readiness of the F3-aware V2 RPC APIs. By embedding an F3 snapshot into the current Filecoin CAR snapshot, both can be vastly reduced at the cost of a slightly increased Filecoin CAR snapshot size. -## Specification +## V1 Specification + +We define the existing Filecoin snapshot format as v1 here for future reference. + +Filecoin snapshot v1 is in [CARv1](https://ipld.io/specs/transport/car/carv1/) format. + +The roots array in the `CarHeader` stores the tipset keys of the chain head in the snapshot. + +The data blocks are chain IPLD blocks generated in a deterministic depth-first traversal order during chain export. Thus a snapshot is deterministic for a given chain head and the number of state trees to include. (The details of the chain traversal algorithm can be found in Filecoin node implementations.) + +```go +type CarHeader struct { + version Int + roots [&Any] +} +``` + +## V2 Specification -We propose the below changes to the Filecoin CAR snapshot format. +We propose the below changes to the [V1 Filecoin CAR snapshot format](#v1-specification). - Change CAR root to be a CID that points to a CBOR-encoded [`SnapshotMetadata`](#snapshotmetadata) struct that is stored as the first data block in the CAR. - Store the raw [`F3Data`](#f3data) bytes as the second data block in the CAR when `F3Data != nil` in the metadata. @@ -44,8 +61,8 @@ We propose the below changes to the Filecoin CAR snapshot format. ```go type SnapshotMetadata { Version uint64 // required, format version for SnapshotMetadata. Only "2" is supported since "v1" was implied in the original format that predates `SnapshotMetadata`. - HeadTipsetKey []Cid // required - F3Data *Cid // optional, points to F3Data structure. The only supported codec is "RAW" (0x55). + HeadTipsetKey []Cid // required + F3Data *Cid // optional, points to F3Data structure. The only supported codec is "RAW" (0x55). } ``` @@ -61,12 +78,12 @@ A header block is a CBOR-encoded [`F3SnapshotHeader`](#f3snapshotheader) with a A data block is a CBOR-encoded [`FinalityCertificate`](#finalitycertificate) with a length prefix in the below format: -`[varint-encoded byte length] [CBOR-encoded FinalityCertificate]` +`[varint-encoded byte length of "CBOR-encoded FinalityCertificate"] [CBOR-encoded FinalityCertificate]` Notes: - `FinalityCertificate`s should be ordered by `GPBFTInstance` in ascending order for sequential validation and intermediate power table calculation. This also ensures deterministic generation of F3 snapshot from a given F3 finality certificate chain. -- The first and last `FinalityCertificate` instances should match those in the header, respectively. -- This [CARv1](https://ipld.io/specs/transport/car/carv1)-like format is ideal for dumping blocks via streaming reads as the Header can be loaded first and minimal state is required for ongoing parsing. +- The first and last `FinalityCertificate` instances should match those in the [F3SnapshotHeader](#f3snapshotheader), respectively. +- This [CARv1](https://ipld.io/specs/transport/car/carv1)-like format is ideal for dumping blocks via streaming reads as the [F3SnapshotHeader](#f3snapshotheader) can be loaded first and minimal state is required for ongoing parsing. ### F3SnapshotHeader @@ -174,37 +191,20 @@ type TipSet struct { type TipSetKey = []byte ``` -### Filecoin snapshot v1 format - -We define the existing Filecoin snapshot format as v1 here for future reference. - -Filecoin snapshot v1 is in [CARv1](https://ipld.io/specs/transport/car/carv1/) format. - -The roots array in the `CarHeader` stores the tipset keys of the chain head in the snapshot. - -The data blocks are chain IPLD blocks generated in a deterministic depth-first traversal order during chain export. Thus a snapshot is deterministic for a given chain head and the number of state trees to include. (The details of the chain traversal algorithm can be found in Filecoin node implementations) - -```go -type CarHeader struct { - version Int - roots [&Any] -} -``` - ## Backwards Compatibility -- A Filecoin node should try to read a snapshot CAR in the proposed format. A failure to successfully decode the block referenced as the CAR's single root using the schema presented above, a snapshot reader may fallback to the old format and maintain backward compatibility. Additional failures to decode original snapshot format would indicate a fatal error. +- A Filecoin node should try to read a snapshot CAR in the [v2 format](#v2-specification). If there is a failure to successfully decode the block referenced as the CAR's single root using the schema presented above, a snapshot reader may fallback to the [v1 format](#v1-specification) and maintain backward compatibility. Additional failures to decode original snapshot format would indicate a fatal error. - CLI options for implementations like Forest, Lotus and Venus should remain unchanged to make it transparent to the node users. - The code change in all Filecoin nodes should be shipped with a network upgrade, and the Filecoin snapshot providers should only start publishing with the new format after the mainnet upgrade finishes to avoid potential errors during snapshot import for node users. That is to say: - Before NV27, - - a node reads both v1 and v2 snapshots - - a node generates v1 snapshot by default - - node provides publish and host v1 snapshots. + - node implementations can read v1 snapshots for sure and v2 snapshot reading is being rolled out before the upgrade. + - node implementations generate v1 snapshots by default + - node implementation providers publish and host v1 snapshots - After NV27, - - a node reads both v1 and v2 snapshots - - a node generates v2 snapshot by default - - node providers publish and host v2 snapshots. + - node implementations can read both v1 and v2 snapshots + - node implementations generate v2 snapshots by default + - node implementation providers publish and host v2 snapshots ## Test Cases @@ -235,8 +235,9 @@ Nodes starting from a snapshot should not rely on the certificate exchange proto ## Implementation -Lotus: https://github.com/filecoin-project/lotus/issues/13129 -Forest: +- Lotus: https://github.com/filecoin-project/lotus/issues/13129 +- Forest: + ## Future Work From 11a72919ead87e18d9b00046a682346b89cfd405 Mon Sep 17 00:00:00 2001 From: Steve Loeppky Date: Tue, 1 Jul 2025 21:09:40 -0700 Subject: [PATCH 5/7] biglep@ understanding of additional design considerations and decisions. --- FRCs/frc-0108.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/FRCs/frc-0108.md b/FRCs/frc-0108.md index 9204caf58..c4da557d1 100644 --- a/FRCs/frc-0108.md +++ b/FRCs/frc-0108.md @@ -84,6 +84,9 @@ Notes: - `FinalityCertificate`s should be ordered by `GPBFTInstance` in ascending order for sequential validation and intermediate power table calculation. This also ensures deterministic generation of F3 snapshot from a given F3 finality certificate chain. - The first and last `FinalityCertificate` instances should match those in the [F3SnapshotHeader](#f3snapshotheader), respectively. - This [CARv1](https://ipld.io/specs/transport/car/carv1)-like format is ideal for dumping blocks via streaming reads as the [F3SnapshotHeader](#f3snapshotheader) can be loaded first and minimal state is required for ongoing parsing. +- This is "CARv1-like" but not true CARv1 because data blocks are not content addressed by CIDs. +- The "varint-encoded byte length" prefixes follow the CARv1 format. It is an implementation detail of the CARv1 format that we're bleeding through here. +- We aren't CBOR-encoding all of F3Data to enable streaming with lower RAM requirements. Node implementations are already experienced at streaming CARs, and we didn't want them to have to properly configure/use CBOR encoding/decoding in a streaming fashion. ### F3SnapshotHeader From 412adf0b4d80d706111d26f6c0f79fbfdd3d7f53 Mon Sep 17 00:00:00 2001 From: hanabi1224 Date: Wed, 2 Jul 2025 20:14:58 +0800 Subject: [PATCH 6/7] Apply suggestions from code review Co-authored-by: Rod Vagg --- FRCs/frc-0108.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/FRCs/frc-0108.md b/FRCs/frc-0108.md index c4da557d1..e51471b97 100644 --- a/FRCs/frc-0108.md +++ b/FRCs/frc-0108.md @@ -60,9 +60,10 @@ We propose the below changes to the [V1 Filecoin CAR snapshot format](#v1-specif ```go type SnapshotMetadata { - Version uint64 // required, format version for SnapshotMetadata. Only "2" is supported since "v1" was implied in the original format that predates `SnapshotMetadata`. - HeadTipsetKey []Cid // required - F3Data *Cid // optional, points to F3Data structure. The only supported codec is "RAW" (0x55). + Version uint64 // Required, format version for SnapshotMetadata. + // Only "2" is supported since "v1" was implied in the original format that predates `SnapshotMetadata`. + HeadTipsetKey []Cid // Required + F3Data *Cid // Optional, points to F3Data structure. The only supported codec is "RAW" (0x55). } ``` @@ -231,9 +232,13 @@ Node users should experience faster F3 bootstrapping time and less network bandw ## Product Considerations -Nodes starting from a snapshot should not rely on the certificate exchange protocol to catch up with the F3 data because we expect this will get slower over time. A slow F3 catchup time leads to, e.g. +### Start-up without initial F3 data -- delay in the readiness of F3-aware RPC APIs +Nodes starting from a snapshot should not rely on the certificate exchange protocol to catch up with the F3 data because we expect this will get slower over time. One outcome of a slow F3 catchup time is a delay in the readiness of F3-aware RPC APIs. + +### CAR format expectations + +This change introduces a relatively novel use of the CAR format in that it contains one very large block, much larger than typical blocks found in most CAR containers for use with IPLD data. At the time of this proposal, this block size would be approximately 100 MiB and this will only grow over time. While this is not disallowed by the CAR specification, many CAR processing utilities are built on an assumption of classic IPFS style blocks of more more than approximately 1MiB each. Some CAR tooling may struggle to deal with the new proposed format, although handling CAR data outside of the narrow use-case of snapshort imports on Filecoin nodes is not typical or necessarily recommended. ## Implementation From 8db56b386470afaf4b5b44c281899193012b90b5 Mon Sep 17 00:00:00 2001 From: hanabi1224 Date: Wed, 2 Jul 2025 20:55:36 +0800 Subject: [PATCH 7/7] reasoning F3SnapshotHeader version --- FRCs/frc-0108.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/FRCs/frc-0108.md b/FRCs/frc-0108.md index e51471b97..2e8c9f7b3 100644 --- a/FRCs/frc-0108.md +++ b/FRCs/frc-0108.md @@ -88,6 +88,7 @@ Notes: - This is "CARv1-like" but not true CARv1 because data blocks are not content addressed by CIDs. - The "varint-encoded byte length" prefixes follow the CARv1 format. It is an implementation detail of the CARv1 format that we're bleeding through here. - We aren't CBOR-encoding all of F3Data to enable streaming with lower RAM requirements. Node implementations are already experienced at streaming CARs, and we didn't want them to have to properly configure/use CBOR encoding/decoding in a streaming fashion. +- A Filecoin node should delegate the F3 data to the underlyding F3 package (e.g. `go-f3`) for importing, and the F3 package should provide API for exporting the F3 snapshot bytes in the same format. Changes in the F3 data format should only bump the version in `F3SnapshotHeader` instead of the version in `SnapshotMetadata`. Backward compatibility for importing and exporting F3 snapshots should be maintained by the underlying F3 package(e.g. `go-f3`), hence transparent to Filecoin nodes. ### F3SnapshotHeader @@ -209,6 +210,7 @@ type TipSetKey = []byte - node implementations can read both v1 and v2 snapshots - node implementations generate v2 snapshots by default - node implementation providers publish and host v2 snapshots +- Backward compatibility for importing and exporting F3 snapshots should be maintained by the underlying F3 package(e.g. `go-f3`), hence transparent to Filecoin nodes. ## Test Cases