wip: squash move partition verify (#12)

* init

* avoid recompute current_deadline

* treat empty bitfield as all

* rm useless quote

* add verify

* combine option1 & option2

* fix

* fix

* nit

* mod error

* nit

* fmt

* fix ci

* fix new_deadline_info_from_offset_and_epoch (filecoin-project#1321)

* fix new_deadline_info_from_offset_and_epoch

* remove useless `% policy.wpost_period_deadlines`

* add comment to wpost_challenge_window

* Expand documentation for deal activation and verified claim parameters and results. (filecoin-project#1325)

* fix bug

* add test

* add more test

* Update FVM types (filecoin-project#1330)

* bump versions inline with next helix set

* temp bump to allow helix-onchain filecoin to build its integration tests

* update to new fvm versions

* unpatch helix libraries

* Have VM trait expose a dyn blockstore (filecoin-project#1328)

* wip

* refactor VM trait to return a dyn Blockstore

* refactor miner_withdrawal_tests to not need BS: Blockstore

* VM trait should use abstract primitives (filecoin-project#1329)

* partial fix for review

* adjust test

* use .context_code

* fix for test

* disallow empty partitions

* refactor deadline_available_for_move

* fix for clippy

* minor opt

* only verify_windowed_post once

* mod error msg

* 1. verify_window_post batch by batch
2. move partitions intact, the `expirations_epochs` and `early_terminated` may have a gap of 24H

* fix ci

* chore: specify all dependencies in the workspace (filecoin-project#1341)

- This makes it easier to update them and makes dependency upgrade diffs
smaller.
- Reduces rebase conflicts.
- Reduces accidental duplicate dependencies (different versions).

* mod check for epoch

* Update VM trait (filecoin-project#1340)

* update VM trait to align with anorth/fvm-workbench

* update tests to check invariants on the concrete types

* Revert "update tests to check invariants on the concrete types"

This reverts commit 8351d66.

* wip

* extract policy out of VM trait

* add set circulating supply onto the trait

* use static builtin-actors manifest

* Claim allocations processes batches by sector (filecoin-project#1337)

* Claim allocations processes batches by sector

* review

* restore illegal argument on batch failure

* fmt

* extract VM trait to it's own crate (filecoin-project#1343)

* extract VM trait to it's own crate

* prune vm_api depdendencies

* highlight TODO to prune fil_actors_runtime from the vm_api depdency

* tracking issue for dropping fil_actors_runtime

* Export integration tests (filecoin-project#1345)

* move test related utils and code out of the test_vm crate

* add utils into integration_tests folder

* port authenticate message test

* port withdraw balance test

* batch_onboarding

* batch_onboarding_deals

* change beneficiary tests

* change owner test

* commit_post

* datacap tests

* evm tests

* extend sectors tests

* cargo updates: wip

* market miner withdrawal

* multisig test

* power scenario

* publish deals

* replica update test

* terminate test

* test vm test

* verified claim

* verifreg tests

* add back an accidentally deleted test

* remove unecessary deps from test_vm

* add TODO

* partial review fix

* adjust test

* refactor with Partition::adjust_for_move

* Remove duplicated comments in policy values (filecoin-project#1350)

* Remove unused ComputeDataCommitment from built-in market (filecoin-project#1354)

* Test VM state mutation (filecoin-project#1356)

* make power scenario test portable

* port extend sectors test to run on the dyn VM

* port evm tests

* port placeholder_deploy test

* cleanup and refactor replica update tests

* Initial implementation of new, more convenient Map wrapping HAMT (filecoin-project#1349)

* Initial implementation of new, more convenient Map wrapping HAMT

* MapKey trait

* Use Map2 instead of Map and raw HAMTs in market actor (filecoin-project#1358)

* Reverse import of fil_actors_runtime and vm_api (filecoin-project#1359)

* amend actor type to match fvm impl

* reverse builtin <-> runtime dependency

* hide testing utils behind feature flag

* Remove uses of Map from most actors (filecoin-project#1363)

* Remove uses of Map from most actors

* Use BigIntDe for datacap

* fix: explicitly specify HAMT bitwidths everywhere (filecoin-project#1364)

This _doesn't_ migrate to the new `Map2` interface everywhere, it just
avoids the use of `Hamt::load` and `Hamt::new`, which will be deprecated
in the next release of `fvm_ipld_hamt`.

part of filecoin-project#1346

* Adds CommD as option in return value from ActivateDeals (filecoin-project#1361)

* Optionally return CommD from ActivateDeals, avoid redundant call in ProveReplicaUpdates

* Fix integration tests

* ci: remove the build cache (filecoin-project#1369)

* ci: remove the build cache

It's not helping and is more likely to break builds than to speed them
up. In practice, it actually appears to be slowing them down.

related to filecoin-project#1366

* ci: install rustfilt with the package manager

* ci: disable incremental builds

* Portable Invocation Traces (filecoin-project#1367)

* update InvocationTrace

* portable invocation traces

* simplify expectation checking

* fix replica update test expectations

* Speed up very slow market test (filecoin-project#1373)

* Refactor sector proving methods to a unified activate_deals data activation  (filecoin-project#1370)

* Alternate confirm_sector_proofs_valid

* Post rebase fix

* Refactor

* Fix up

* Add input/output names for data activation types

* Fix plumbing computed CommD back to replica proof verification

* Review Response

* Revert deal id removal for tests

* Adapt test for bad expiry value during CSPV to new behavior

* Fail cleanly in case sector can be activated with zero duration

* fmt

* Clippy

---------

Co-authored-by: zenground0 <[email protected]>
Co-authored-by: Alex North <[email protected]>

* Refactor replica update sector info to new method (filecoin-project#1376)

* Refactor replica update validation to new method

* Refactor replica update sector info to new method

* review

* Refactor: extract precommit validation and svi conversion to shared method (filecoin-project#1375)

* Factor out precommit check and svi conversion

* Review Response

---------

Co-authored-by: zenground0 <[email protected]>

* chore: remove support for verifying single seals (filecoin-project#1382)

We don't use this.

* test: make the test_utils module available in runtime tests (filecoin-project#1383)

test: make the test_utils moudle available in runtime tests

* share the language with FIP

* deadline_available_for_move => ensure_deadline_available_for_move

* add some doc comment

* more renaming

* more renaming

* rename + merge master

* mod wording

* fix test

* Remove go-interop feat from the AMT crate (filecoin-project#1390)

* renaming in test

* chore: update eth deps and remove default features (filecoin-project#1392)

replaces filecoin-project#1384

* chore: upgrade actor bundler to use the latest blockstore (filecoin-project#1368)

* chore: upgrade actor bundler to use the latest blockstore

We were importing both a v0.1.0 and a v0.2.0 blockstore from the FVM,
now we only import the latter.

This also upgrades to the latest clap version (FWIW).

* chore: upgrade bitfield to use the latest blockstore

Apparently, just the bundler wasn't enough.

* test: move all integration tests into one target (filecoin-project#1397)

This makes compilation _much_ faster.

* EVM: propagate return data when contract creation fails (filecoin-project#1291)

feat: propegate return data when contract creation fails

And clear the return data otherwise.

* replace simple_qa_power with extensible flags (filecoin-project#1395)

* replace simple_qa_power with extensible flags

* use u32 for SectorOnChainInfoFlags

* chore: update the fvm (filecoin-project#1405)

* chore: update the fvm

This doesn't add support for synthetic porep, but splits:

filecoin-project#1335

Into two PRs.

* finish update

* I guess patches aren't enough

* fix evm tests

We now need to explicitly specify the sha3/ripemd features when
testing (it was removed from shared).

* fix clippy

* switch back to helix master branch

so that helix can pass CI again

* use released helix crates

* Add proof types for synthetic porep (FIP-0059) (filecoin-project#1409)

* finish update

* I guess patches aren't enough

* fix evm tests

We now need to explicitly specify the sha3/ripemd features when
testing (it was removed from shared).

* feat: adds support for Synthetic PoRep
feat: updates to use latest fvm releases

* style: rust fmt

* fix: add proper seal proof variant count

* fix: apply review feedback

* style: rust fmt

---------

Co-authored-by: Steven Allen <[email protected]>

* nit: alloc `expiration_groups` after `declared_expirations` is set (filecoin-project#1413)

alloc expiration_groups after declared_expirations is set

* dispatch: improve dispatch macro and allow specifying codecs (filecoin-project#1412)

* dispatch: make it possible to specify the param/return codec

And specify it for two EVM methods that need to pass "reachable" CIDs:

1. The `bytecode` return value.
2. The `invoke_contract_delegate` parameters.

* dispatch: auto detect "raw" dispatch

Instead of forcing the user to specify `[raw]`, just auto detect it
based on the method parameters.

* dispatch: test changes

* address feedback

* nit: avoid recompute `quantize_up` in loops (filecoin-project#1415)

avoid recompute quantize_up

* apply alex's idea of not re-quantizing at all.

* 1. forbid moving when there're early terminations
2. remove `adjust_for_move`

* rm anyhow::Ok

* minor optimization by observing that partition `faulty_power` should be zero when `faults` is empty

* adjust find_sectors_by_expiration for not re-quantizing

* add test

* fix for review

* add a comment about not re-quantizing when moving expirations_epochs

* minor optimization

* avoid scanning the same range twice

* 1. review fix
2. add check for `max_partitions_per_deadline`

* fix comment

* use with_context_code

---------

Co-authored-by: zhiqiangxu <[email protected]>
Co-authored-by: Alex North <[email protected]>
Co-authored-by: Alex Su <[email protected]>
Co-authored-by: Steven Allen <[email protected]>
Co-authored-by: Alex Su <[email protected]>
Co-authored-by: ZenGround0 <[email protected]>
Co-authored-by: zenground0 <[email protected]>
Co-authored-by: Hubert <[email protected]>
Co-authored-by: nemo <[email protected]>

Author: 8 people

.github/workflows/ci.yml

@@ -9,6 +9,7 @@ on:
 
 env:
   RUSTFLAGS: -Dwarnings
+  CARGO_INCREMENTAL: 0 # Speeds up the build (no cache) and reduces disk space!
 
 jobs:
   rustfmt:
@@ -32,11 +33,6 @@ jobs:
     steps:
     - name: Checking out
       uses: actions/checkout@v3
-    - name: Setting up cache
-      uses: pl-strflt/rust-sccache-action@v1
-      env:
-        SCCACHE_CACHE_SIZE: 2G
-        SCCACHE_DIR: ${{ github.workspace }}/.cache/sccache
     - name: Running tests
       run: |
         cargo test --locked --all --no-fail-fast --exclude=fil_builtin_actors_bundle
@@ -49,12 +45,6 @@ jobs:
     steps:
     - name: Checking out
       uses: actions/checkout@v3
-    - name: Setting up cache
-      uses: pl-strflt/rust-sccache-action@v1
-      env:
-        SCCACHE_CACHE_SIZE: 2G
-        SCCACHE_DIR: ${{ github.workspace }}/.cache/sccache
-        CACHE_SKIP_SAVE: true
     - name: Writing bundle
       env:
         BUILD_FIL_NETWORK: ${{ matrix.network }}
@@ -67,17 +57,10 @@ jobs:
     steps:
     - name: Checking out
       uses: actions/checkout@v3
-    - name: Setting up cache
-      uses: pl-strflt/rust-sccache-action@v1
-      env:
-        SCCACHE_CACHE_SIZE: 2G
-        SCCACHE_DIR: ${{ github.workspace }}/.cache/sccache
-      with:
-        shared-key: coverage
     - name: Put LLVM tools into the PATH
       run: echo "$(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/bin" >> $GITHUB_PATH
     - name: Install demangler
-      run: cargo install rustfilt
+      run: sudo apt-get install -y rustfilt
     - name: Create coverage report
       env:
         # Make sure that each run of an executable creates a new profile file,