perf(l1): decode engine_newPayload transactions in parallel

[ilitteri]

Motivation

engine_newPayload runs on the block-import critical path: the consensus client
waits for the response, and the handler executes before returning VALID. Profiling
that path showed a small but real amount of work sits before execution and is not
counted by the block-processing pipeline's timing (gigagas / total_ms): turning the
payload into a Block. Part of that is decoding the transactions.

ExecutionPayload::into_block decoded the payload's transactions sequentially, even
though each transaction is an independent RLP blob.

Description

Decode the payload's transactions in parallel with rayon's par_iter. The result is
identical to the sequential decode:

• par_iter().map().collect::<Result<Vec<_>, _>>() is order-preserving and
  short-circuits on the first decode error, so the decoded Vec<Transaction> and any
  error are exactly what the sequential path produced.
• The eip-8025 (stateless / zkVM guest) build keeps the deterministic sequential path,
  mirroring the existing get_transactions_with_sender.

Scope is intentionally tiny: only the decode loop in into_block changes.

How to test

cargo test -p ethrex-rpc --lib types::payload — the deserialize_payload_into_block
test decodes a mixed legacy + EIP-1559 payload and now also asserts that the parallel
decode matches a sequential decode (same order, same content).

Notes

This is a pure, deterministic parallelization (no consensus behavior change). A follow-up
change computes the transactions root directly from the payload's raw canonical bytes
(skipping a decode→re-encode round-trip); it is kept separate as it warrants its own review.

[github-actions]

Lines of code report

Total lines added: 31
Total lines removed: 0
Total lines changed: 31

Detailed view

+-----------------------------------------------+-------+------+
| File                                          | Lines | Diff |
+-----------------------------------------------+-------+------+
| ethrex/crates/networking/rpc/types/payload.rs | 379   | +31  |
+-----------------------------------------------+-------+------+

[github-actions]

🤖 Kimi Code Review

Overall Assessment: The change is correct and well-structured. Parallel RLP decoding of transactions is safe because they're independent blobs, and Rayon preserves order in collect(). The feature-gating for deterministic ZK builds is appropriate.

Specific Feedback:

1. Dependency Management (Cargo.toml)
Ensure rayon is declared as an optional dependency that's excluded when eip-8025 is enabled to minimize binary size for ZK targets:

rayon = { version = "1.x", optional = true }

[features]
default = ["rayon"]
eip-8025 = []

Verify this is handled in the Cargo.toml changes not shown in the diff.

2. Test Clarity (payload.rs:396-420)
The test validates that parallel and sequential decoding produce identical results, but the behavior depends on the compile-time feature flag. When eip-8025 is enabled, both paths are sequential (trivially passing). Consider explicitly testing both paths or documenting that the test validates the current configuration:

// Option: Explicitly test parallel path regardless of feature flags
#[cfg(not(feature = "eip-8025"))]
{
    // parallel decode test
}

3. Parallelism Threshold (Optimization)
For small blocks (e.g., < 16 transactions), Rayon's threadpool overhead may exceed the savings. Consider adding a fallback:

if self.transactions.len() > 16 {
    // use par_iter()
} else {
    // use iter()
}

This is optional and should be guided by profiling on the engine_newPayload path.

4. Comment Accuracy (payload.rs:118)
"eip-8025" does not appear to be a standardized EIP. If this is an internal codename, consider clarifying in the comment:

// eip-8025: internal feature flag for stateless/ZK guest builds (deterministic execution)

Security Verification:

• Order preservation: Rayon's collect() guarantees order preservation (Line 125), ensuring transaction root calculation remains deterministic across architectures.
• Error handling: Short-circuiting on first decode error (Line 126) is correct for consensus validation.
• Memory safety: No shared mutable state between parallel workers; each encoded_tx.decode() operates on independent Bytes.

Nitpick:
The use rayon::prelude::*; import inside the cfg block (Line 121) is clean and prevents unused import warnings. Good practice.

Approval: LGTM pending verification of the Cargo.toml dependency gating mentioned in Point 1.

---

Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

[github-actions]

🤖 Codex Code Review

1. crates/networking/rpc/types/payload.rs: par_iter().collect::<Result<Vec<_>, _>>() changes the invalid-payload behavior on engine_newPayload* from effectively fail-fast to “decode as much as the scheduler already fanned out.” If the first transaction is malformed but the payload carries a large tx list, the node can now burn substantially more CPU before rejecting the block. Because this path is consensus-client driven and latency-sensitive, that is a real DoS/perf regression on bad inputs. A second side effect is that, if multiple txs are malformed, the surfaced RLPDecodeError is no longer deterministic. I would keep the sequential path for small/invalid-first payloads, or gate parallelism behind a size threshold plus a clear decision that nondeterministic error selection is acceptable.

Beyond that, the happy-path change looks mechanically sound: par_iter() on an indexed Vec should preserve transaction order, and the new test usefully checks order/content preservation. I was not able to run cargo test here because the environment blocks rustup from creating temp files under /home/runner/.rustup/tmp (read-only filesystem).

---

Automated review by OpenAI Codex · gpt-5.4 · custom prompt

[Copilot]

Pull request overview

This PR parallelizes transaction decoding when converting an ExecutionPayload into a Block, reducing time spent in engine_newPayload’s pre-execution critical path while keeping a sequential path for eip-8025 builds.

Changes:

• Decode ExecutionPayload transactions in parallel via rayon::par_iter() (sequential under eip-8025).
• Update the payload deserialization test to assert that parallel decoding preserves transaction order and content vs a sequential reference decode.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[greptile-apps]

Greptile Summary

This PR parallelizes transaction decoding in ExecutionPayload::into_block using Rayon's par_iter, targeting the engine_newPayload critical path where the consensus client blocks waiting for a response. The eip-8025 (stateless/zkVM guest) feature gate preserves the deterministic sequential path.

• The parallel decode is guarded by #[cfg(not(feature = \"eip-8025\"))] and falls back to sequential iteration under that feature, matching the existing pattern used in get_transactions_with_sender.
• rayon was already a workspace dependency in ethrex-rpc/Cargo.toml, so no new dependency is introduced.
• The updated test encodes both a sequential reference decode and the parallel result back to canonical bytes, asserting order and content equality.

Confidence Score: 5/5

Safe to merge — the change is a pure data-parallel decode with no behavioral difference on the happy path, and the sequential fallback for eip-8025 builds is intact.

The modification is tightly scoped to one decode loop in into_block. Rayon's par_iter().collect::<Result<Vec<_>, _>>() is order-preserving and returns the first error by index, so both the success and error paths are semantically identical to the prior sequential code. The rayon crate was already a workspace dependency. The test now explicitly cross-checks the parallel result against a sequential reference decode, providing direct regression coverage for order correctness.

No files require special attention.

Important Files Changed

Filename	Overview
crates/networking/rpc/types/payload.rs	Transactions in into_block are now decoded in parallel via rayon::par_iter (guarded by #[cfg(not(feature = "eip-8025"))]); the test is extended to assert order and content parity between the parallel and a sequential reference decode.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CL as Consensus Client
    participant RPC as engine_newPayload Handler
    participant P as ExecutionPayload::into_block
    participant R as Rayon Thread Pool
    participant VM as Block Execution

    CL->>RPC: engine_newPayload(payload)
    RPC->>P: payload.into_block(...)
    alt not eip-8025 feature
        P->>R: par_iter().map(decode).collect()
        R-->>P: "Vec<Transaction> (order-preserved)"
    else eip-8025 feature
        P->>P: iter().map(decode).collect()
    end
    P-->>RPC: "Block { header, body }"
    RPC->>VM: execute block
    VM-->>RPC: result
    RPC-->>CL: VALID / INVALID

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CL as Consensus Client
    participant RPC as engine_newPayload Handler
    participant P as ExecutionPayload::into_block
    participant R as Rayon Thread Pool
    participant VM as Block Execution

    CL->>RPC: engine_newPayload(payload)
    RPC->>P: payload.into_block(...)
    alt not eip-8025 feature
        P->>R: par_iter().map(decode).collect()
        R-->>P: "Vec<Transaction> (order-preserved)"
    else eip-8025 feature
        P->>P: iter().map(decode).collect()
    end
    P-->>RPC: "Block { header, body }"
    RPC->>VM: execute block
    VM-->>RPC: result
    RPC-->>CL: VALID / INVALID

Loading

_{Reviews (1): Last reviewed commit: "Decode engine_newPayload transactions in..." | Re-trigger Greptile}