Synthetic PoRep - no buffer between seal and commit

[Kubuxu]

Problem

The PoRep protocol today requires miners to store 12x sector size of layers data after it is created during the replication step until the sector is proven in the on-chain Commit step.
This leads to major inefficiencies where storage space needed to perform replication is occupied for a prolonged period of time by the layers data. This also prevents some miners from efficiently using aggregation.

@Kubuxu @nicola @lucaniz @rosariogennaro

Solution

Synthetic PoRep achieves reduction in used up space by reducing the set of challenges that might be chosen during the interactive Commit step from all possible challenges to some predetermined number that is feasible to precompute.

We propose a Synthetic PoRep protocol where:

• A miner can complete the challenge generation and vanilla proof computation before performing PreCommit on-chain thus removing layers data before the sector is pre-committed on-chain.
• The GPU cost for SNARK generation during Commit is not significantly increased

Protocol Flow

The following section describes the high-level flow of the protocol. Differences between currently deployed PoRep and Synthetic PoRep are limited to challenge generation and additional capabilities for the miner.

1. Starting point
   1. We assume there is a sector S for which the miner completed PreCommit1 and PreCommit2 computations.
   2. This sector is not listed on-chain.
   3. The miner possesses knowledge or CommR and CommD of that sector (acquired in PreCommit2 step) and the layers needed to generate CommR
2. Miner generates “synthetic” challenges from CommR
   1. Based on the CommR, the miner generates a list of SynthChallengesNumber (currently ~270k) challenges Ch_all.
   2. Knowing all the challenges, the miner computes responses for them which take the form of SynthChallengesNumber vanilla proofs and saves them for future use.
3. Miner can remove layers data
   1. As the miner knows responses to all possible challenges that will be asked in the interactive step, he can remove the layers data which is needed to respond to challenges.
4. Miner publishes “PreCommitsSector”
   1. Using the same flow as today miner submits the sector for PreCommit.
   2. This establishes when in the future the randomness for interactive response will be known (PreCommitChallengeDelay)
5. Miner generates and publishes ProveCommitSectors proof
   1. Miner waits PreCommitChallengeDelay (~150 epochs).
   2. The randomness revealed at PreCommitEpoch+PreCommitChallengeDelay selects VerifiedChallengesNumber (176) challenges to be verified on-chain from Ch_all.
   3. Miner takes the VerifiedChallengesNumber vanilla proofs which were generated earlier corresponding to selected challenges and computes SNARK proofs of these challenges.
   4. Miner publishes the ProveCommit either in individual or aggregated form.
6. Chain verifies proof
   1. Using interactive randomness as a seed chain generates VerifiedChallengesNumber challenges by selecting VerifiedChallengesNumber indices out of SynthChallengesNumber and computing them.
   2. Generated challenges are fed into proof verification.

Parameter Selection

Based on the current analysis following parameters are under consideration:

• SynthChallengesNumber = ~271,000 with VerifiedChallengesNumber= 176 - verification cost similar to today
• SynthChallengesNumber = ~84,000 with VerifiedChallengesNumber=198 - verification and GPU cost increased by 12.5%

[ZenGround0]

Thanks for the explanation. I have a few basic questions

1. Looking at the red lightning bolt challenges that the miner stores it looks like each challenge consists of some subset of the 12 layers data following a path through the layer graph. Is that true or is it more complicated than this?
2. What percentage of the current storage overhead does storing ~270k challenges take?
3. What's the summary for why grinding synthetic challenges by changing CommR doesn't help the miner? (sealing is expensive, SynthChallengesNumber is high etc)
4. There is some discussion of GPU cost of PoRep maybe going up with this construction. Is this just a function of increasing total verified challenges?
5. Does changing the number of verified challenges in PoRep (or any other change here) require new trusted setup params?

[Kubuxu]

Looking at the red lightning bolt challenges that the miner stores it looks like each challenge consists of some subset of the 12 layers data following a path through the layer graph. Is that true or is it more complicated than this?

That is roughly it, this is what proofs code calls "vanilla proofs". They are nodes in the DAG structure of DRG "graph" proving that you know the path from the root to given leaves.

What percentage of the current storage overhead does storing ~270k challenges take?

It should be around 26GiB which is ~7% of what needs to be stored today (11 layers ~ 350GiB), same as today with the 12x layers data, it can be removed after sector is committed on chain.

What's the summary for why grinding synthetic challenges by changing CommR doesn't help the miner? (sealing is expensive, SynthChallengesNumber is high etc)

The SynthChallengesNumber is chosen such that grinding CommR is infeasible for an attacker (2^80 grinding attempts to reduce the security of PoRep by 1bit). Grinding CommR in itself is not an expensive process.'

There is some discussion of GPU cost of PoRep maybe going up with this construction. Is this just a function of increasing total verified challenges?

Yes. Currently, we verify challenges in sets of 22 per SNARK proof and miners need to produce 8 "partition" proofs each proving a different set of challenges. We could reduce the number of Synthetic Challenges while maintaining the same protocol security by increasing the number of verified challenges (in increases of 22) but this incurs an additional cost of producing and verifying the Commit proof.

Does changing the number of verified challenges in PoRep (or any other change here) require new trusted setup params?

Increasing the number of verified challenges would require miners to produce more proofs following the same scheme, which does not require a trusted setup.

The Synthetic PoRep protocol itself changes the manner in which challenges are generated, which also does not require circuit changes (chosen challenges are fed to the SNARK), and thus doesn't require another phase 2 trusted setup.

[nicola]

What percentage of the current storage overhead does storing ~270k challenges take?

Back of the envelope calculation

Assuming 275k challenges (number seems could go down). If you were to naively store all you need (there is a lot of repeated data in merkle trees)

For each challenge, we need to store 104,128 bytes

• commD tree (17 * 30 * 32bytes)
• commR tree (17 * 10 * 7 mtree siblings * 32bytes)
• commC tree (17 * 10 * 7 mtree siblings * 32bytes)
• commD data (11 layers * 16 * 32bytes)
• commR data (11 layers * 1 * 32 bytes)
• commC data (11 layers * 17 * 32 bytes)

For a total of ~26 GiB instead of 448 GiB

[dd45e640b42e6da7da96faee3996ef7c]

This also prevents some miners from efficiently using aggregation.

i think you mean batching

[jennijuju]

This also prevents some miners from efficiently using aggregation.

i think you mean batching

I believe it is referring to proof aggregation for ProveCommit - as right now miners need to keep huge sizes of files for C1 and occupying resources -> it prevents miners from aggregate the maximum/larger amount of proof as they can

[dd45e640b42e6da7da96faee3996ef7c]

This also prevents some miners from efficiently using aggregation.

i think you mean batching

I believe it is referring to proof aggregation for ProveCommit - as right now miners need to keep huge sizes of files for C1 and occupying resources -> it prevents miners from aggregate the maximum/larger amount of proof as they can

yeah - read it again, it seems not like it will help with precom batching

[Kubuxu]

seems not like it will help with precom batching

It will help with PreCommit batching as well, as you don't need to keep the 448GB around until Commit.
Unless I'm missing another issue with PreCommit batching.

[dd45e640b42e6da7da96faee3996ef7c]

It will help with PreCommit batching as well, as you don't need to keep the 448GB around until PreCommit and then Commit.
Unless I'm missing another issue with PreCommit batching.

if you miss something we are all doomed 🥇

will the 450Gb even be created and stored or does it just eliminates them completely "on the fly"? if they get created when exactly can it be deleted?

[Kubuxu]

The 450GB are created as part of the sealing process, after sealing is completed, the Synthetic Challenges can be generated, vanilla proofs for these challenges can be computed (the 26GB Nicola is talking about) and then layers data can be removed leaving you with whatever amount of data is left after sector is Committed on-chain + the 26GB of vanilla proofs. The vanilla proofs can then be removed when the sector is Committed.

[steven004]

This looks great. With this research and implementation in progress, one thing I would like to bring up for discussion:
Are we considering this as another version of PoRep co-existing with the current one, or a new version to replace the current one?

I would suggest this can be discussed in our next co-dev meeting. @arajasek

[ozhtdong]

I assume the extra 26GB vanilla proof will require extra time for seal, so basically it's a tradeoff between seal time and disk space, would be great if we can choose between retain layer files or remove layers but with extra vanilla proof generation step, considering there might be different hardware setup with enough disk for layer files.

[Kubuxu]

From my estimates, the SynthChallengesNumber (~270k proofs or ~26GB) of vanilla proofs would take around 6 to 12 minutes to complete.
If there is an interest in it, the Synthetic PoRep can function as the current one. In essence, you just skip the synthetic challenge generation and bulk vanilla proofs generation and instead you wait for the randomness, generate the VerifiedChallengesNumber challenges and prove them. The challenge generation function then changes a bit but it is insignificant (~16 extra invocations of SHA for whole PoRep during challenge generation, today we invoke SHA 180 times to compute challenges for PoRep).