Skip to content

feat(spice): support the near-indexer under decoupled execution#16010

Draft
darioush wants to merge 1 commit into
near:masterfrom
darioush:spice/indexer-support
Draft

feat(spice): support the near-indexer under decoupled execution#16010
darioush wants to merge 1 commit into
near:masterfrom
darioush:spice/indexer-support

Conversation

@darioush

@darioush darioush commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Makes the near-indexer stream real data under spice (decoupled execution) instead of empty per-shard shells, and fills the receipt-persistence gap that surfaced once it did.

  • Indexer: build_streamer_message no longer short-circuits under spice; the existing per-chunk loop runs unchanged. Under spice the view client already resolves Finality::None to the first executed ancestor, so the streamer only ever builds messages for already-executed blocks.
  • Receipt bodies: spice chunks are tx-only, so the chain path's partial-chunk save never writes produced-receipt bodies to DBCol::Receipts, leaving get_receipt returning None for incoming receipts (e.g. gas refunds) that the indexer needs. The spice chunk executor now persists produced-receipt bodies by id, with a matching GC decrement (gc_spice_outgoing_receipt_bodies) and store-validator accounting so the DBCol::Receipts refcount stays balanced. All three legs are spice-gated per block; the non-spice path is unchanged.
  • Tests: enables the seven previously spice-ignored test_indexer_* tests (three now anchor on the execution head rather than the consensus head()), and adds test_spice_outgoing_receipt_bodies_gc covering persist → GC with validate_store on both sides.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables the near-indexer to stream real execution data when running with SPICE (decoupled execution) by removing the SPICE short-circuit in streamer message construction. To support indexing under SPICE’s tx-only chunks, it also persists produced (outgoing) receipt bodies into DBCol::Receipts and adds corresponding GC + store-validator accounting to keep refcounts balanced. Test-loop indexer tests are re-enabled under SPICE, and a new GC test is added to cover the produced-receipt-body persist → GC lifecycle.

Changes:

  • Remove the SPICE early-return in build_streamer_message so per-chunk indexing runs under SPICE.
  • Persist SPICE-produced outgoing receipt bodies into DBCol::Receipts, and GC/decrement them in tandem with OutgoingReceipts cleanup, including store-validator refcount accounting.
  • Re-enable previously SPICE-ignored indexer tests and add a targeted GC test for outgoing receipt bodies under SPICE.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test-loop-tests/src/tests/processed_receipts_gc.rs Adds SPICE-only test verifying produced receipt bodies are persisted and later GC’d with refcount validation.
test-loop-tests/src/tests/indexer.rs Re-enables indexer tests under SPICE and adjusts height anchoring to use execution head where needed.
core/store/src/adapter/chain_store.rs Adds ChainStoreUpdateAdapter::save_receipt to centralize receipt-body refcount persistence.
chain/indexer/src/streamer/mod.rs Removes SPICE short-circuit so streamer messages include full per-chunk data under SPICE.
chain/client/src/spice/chunk_executor_actor/per_shard.rs Persists SPICE-produced outgoing receipt bodies into DBCol::Receipts during chunk postprocessing.
chain/chain/src/store/mod.rs Switches chain-path receipt persistence to call the new adapter helper.
chain/chain/src/store_validator/validate.rs Adds SPICE-gated validation accounting for OutgoingReceiptsReceipts refcounts.
chain/chain/src/store_validator.rs Hooks OutgoingReceipts into store validation and ensures validation order accounts for it before Receipts.
chain/chain/src/garbage_collection.rs Adds SPICE-gated GC decrement for produced outgoing receipt bodies prior to deleting OutgoingReceipts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread chain/client/src/spice/chunk_executor_actor/per_shard.rs
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.47059% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.06%. Comparing base (466bb50) to head (b74f594).
⚠️ Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
chain/chain/src/store_validator/validate.rs 57.89% 8 Missing ⚠️
chain/chain/src/garbage_collection.rs 87.50% 0 Missing and 2 partials ⚠️
chain/chain/src/store_validator.rs 75.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #16010      +/-   ##
==========================================
+ Coverage   72.66%   73.06%   +0.39%     
==========================================
  Files         952      856      -96     
  Lines      205109   186724   -18385     
  Branches   205109   186724   -18385     
==========================================
- Hits       149048   136428   -12620     
+ Misses      51075    45944    -5131     
+ Partials     4986     4352     -634     
Flag Coverage Δ
pytests-nightly 1.23% <0.00%> (+0.13%) ⬆️
unittests 69.69% <50.98%> (+0.10%) ⬆️
unittests-nightly 69.69% <50.98%> (+0.08%) ⬆️
unittests-spice 65.44% <43.13%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants