feat(proposervm): dedup inner block bytes when the inner VM retains accepted blocks#5629
Draft
containerman17 wants to merge 6 commits into
Draft
feat(proposervm): dedup inner block bytes when the inner VM retains accepted blocks#5629containerman17 wants to merge 6 commits into
containerman17 wants to merge 6 commits into
Conversation
…ccepted blocks The proposervm stores each accepted block's full container, including the inner block bytes that the inner VM (e.g. subnet-evm) already persists in its own database. On FIFA mainnet (751k blocks) this duplicate copy is 1.36 GiB, 19.7% of the subnet's storage. Inner VMs now declare a capability in their InitializeResponse (retains_accepted_blocks, additive proto3 field): the VM durably retains every accepted block's bytes, retrievable via GetBlock, forever. When the capability is set, the proposervm stores accepted blocks without their inner bytes (state codec v1: stripped block + inner block ID) and reconstructs the exact original bytes on read via the inner VM's GetBlock. subnet-evm declares it; no configuration is involved anywhere. Safety: every stripped block is round-tripped at write time and falls back to full v0 storage on any mismatch (dedup_blocks_fallback metric), and the reconstructed block ID is asserted against the store key on read. Both codec versions are always readable, so mixed databases from upgrades work. Old binaries cannot read v1 records: downgrade after running this code fails chain creation with "unknown codec version". Verified on a mainnet L1 (TixChain, 2834 blocks) with no subnet or chain config: capability engages on its own, 2834/2834 blocks stored deduplicated with 0 fallbacks, block store 7.14 MiB -> 1.51 MiB (79% smaller), clean restarts, and a peer-style GetAncestors walk of the whole chain returns byte-identical containers to a master-built node.
9893762 to
cde9d7b
Compare
coreth never deletes accepted block bodies: state pruning (offline pruning included) only touches the state trie, and only rejected blocks are removed from the chain database. It therefore satisfies the RetainsAcceptedBlocks contract the same way subnet-evm does, with the identical state-sync caveat. The method is declared on the atomic VM wrapper because that is the VM the proposervm wraps: atomicvm.VM embeds the extension.InnerVM interface, which does not include RetainsAcceptedBlocks, so declaring it on evm.VM alone would not surface. coreth runs in-process, so the optional Go interface (not the plugin proto capability) is the path that engages.
A write-time round-trip mismatch has no legitimate data-dependent cause (the check is pure in-memory container mechanics, the inner VM is not involved), so it can only be a strip/restore code bug for some container shape. Counting it was too quiet: nodes would silently keep double storing. Hard-failing PutBlock was considered and rejected because a shape-dependent bug would halt every upgraded node on the same block, and a storage feature must not be able to halt consensus. So the fallback now degrades loudly: tryStripBlock reports the cause (strip error, restore error, byte mismatch, or marshal error) and PutBlock logs it at error level with the block ID, next to the existing dedup_blocks_fallback counter. The logger is plumbed from the chain context through state.NewMetered and is nil-safe like the metric fields. Tests now explode on an unexpected fallback: the dedup round-trip test asserts dedup_blocks_fallback stays 0, and a new test drives the fallback branch with a block whose bytes cannot round-trip, asserting the counter fires and the record falls back to the legacy full format.
…ner block is missing An unclean shutdown between the proposervm accepting a block and the inner VM durably accepting it leaves the proposervm's last accepted above the inner VM's. That state is exactly what repairAcceptedChainByHeight exists to roll back, but with inner-block deduplication its first step, reading the last accepted block to learn its height, needs the missing inner block and failed, so chain creation failed permanently. Since the proposervm commits strictly before the inner VM, a deduplicated tip whose inner block is unavailable can only be above the inner VM's last accepted height. Treat that read failure as proof and proceed with the existing rollback without reading the tip. The same path also covers a node killed during state sync (the state summary accepts the outer block before the inner VM has synced) and an inner chain database that was deleted out from under the proposervm.
…capability' into containerman17/proposervm-dedup-capability
containerman17
added a commit
that referenced
this pull request
Jul 10, 2026
containerman17
added a commit
that referenced
this pull request
Jul 11, 2026
# Conflicts: # vms/proposervm/state/block_state.go # vms/proposervm/state/block_state_test.go # vms/proposervm/state/state.go # vms/proposervm/state/state_test.go # vms/proposervm/vm.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this should be merged
The proposervm persists each accepted block's full container, including the inner block bytes that the inner VM already persists in its own database, so every subnet block is stored twice. Measured on a 751k-block mainnet L1 (FIFA): the duplicate inner bytes are 1.36 GiB, 19.7% of that subnet's total storage (proposervm store 1.755 GiB, of which 1.385 GiB is inner bytes). This removes the second copy with no configuration.
How this works
Inner VMs declare a capability in their
InitializeResponse: a new additive proto3 fieldretains_accepted_blocks = 6meaning the VM durably retains every accepted block's bytes, retrievable viaGetBlock, forever. NoRPCChainVMProtocolbump is required (additive field, defaults false). The capability surfaces through a small optional interface (block.RetainsAcceptedBlocksVM), is set byrpcchainvm.VMServerfrom the served VM, exposed byVMClient, and forwarded throughtracedvm. subnet-evm and coreth (the C-chain) implement it returning true: in both VMs accepted bodies are never deleted (state pruning, offline pruning included, only touches the state trie; only rejected blocks are removed). On the C-chain the method is declared by the atomic VM wrapper, since that is the VM the proposervm wraps and it embeds the inner VM as an interface, so a declaration on the innerevm.VMalone would not surface; coreth runs in-process, so the Go interface rather than the proto field is what engages there.The proposervm checks the capability after the inner VM initializes and, when set, stores accepted blocks in a new state codec version (v1): the block bytes with the inner bytes stripped, plus the inner block ID. Reads reconstruct the exact original container by fetching the inner block from the inner VM. Both codec versions are always readable, so databases with mixed records (normal after an upgrade) work; existing v0 records are not rewritten.
Safety, in depth:
dedup_blocks_fallbackmetric (alongsidededup_blocks_stored). Hard-failing the write instead was rejected: a shape-dependent bug would halt every upgraded node on the same block, and a storage feature must not be able to halt consensus.How this was tested
dedup_blocks_fallbackstays 0, and a dedicated test drives the fallback branch (a block whose bytes cannot round-trip) asserting the counter fires and the record is stored in the legacy full format.inner VM retains accepted blocks; storing blocks without inner bytes),dedup_blocks_stored2834,dedup_blocks_fallback0, all records codec v1, clean restart, historical RPC reads fine.GetBlockhop per block): a p2p test peer walked the full 2835-block chain against both nodes. Master-built node: 0.11 s cold, 0.05 s warm. Dedup node: 1.15 s cold (about 0.4 ms per block for the inner hop, until the proposervm block cache warms), 0.05 s warm. Both nodes served byte-identical containers (7,187,477 bytes total).graft/coreth/plugin/evm/atomic/vm,vms/proposervm). The proposervm side is chain-agnostic, the same write/read path exercised end to end above; the C-chain declaration itself is a constant method guarded by a compile-time interface assertion, and the write-time round-trip fallback applies there like everywhere else.error while creating new snowman vm failed to repair accepted chain by height: failed to get last accepted block: unknown codec version. The node stays up, nothing is written, and re-upgrading works. avalanchego currently has no database version stamp at startup (thev1.4.5in the db path is only a folder name), so making future downgrades fail with a friendlier message would be new node-level machinery; left as future work.Need to be documented in RELEASES.md?
Yes: modifies the proposervm database format for chains whose VM declares
retains_accepted_blocks(subnet-evm and the C-chain); downgrade to earlier versions is not supported after running this code.