Optimize UInt256 logical shifts with a portable funnel shift

[kamilchodola]

Summary

UInt256.Lsh/Rsh are rewritten as a single portable funnel shift. There is no architecture gate and no second implementation.

• The carry term is written (lo >> 1) >> (63 - bitShift). That equals lo >> (64 - bitShift) but yields 0 when bitShift is 0, so whole-word counts need no separate path. The (n % 64) == 0 switch, the Lsh64/Lsh128/Lsh192 and Rsh64/Rsh128/Rsh192 helpers and NativeLsh/NativeRsh are all removed.
• Only the word offset is still branched on, as a compare chain rather than a switch. A 4-way switch here compiles to a jump table reached by an indirect jmp; that predicts perfectly when a benchmark fixes the shift count per case but not on a mixed count stream, and Dynamic PGO cannot reorder it.
• The result is written in one store of the width callers read it back at.
• Negative-count, out-of-range and aliasing semantics are unchanged.

Head history: 8baac47 (the original X86Base.X64-gated implementation) is preserved, main is merged in, and 71be7f6 replaces the implementation. The net diff against main is the funnel rewrite only.

Why the gated implementation was replaced

The previous approach kept origin/main's body as a second implementation behind X86Base.X64.IsSupported, but its own fast path contained no intrinsics, so the gate bought nothing except a duplicate body to maintain.

More importantly, its wordShift switch arms build new UInt256(...), which at the time wrote the result as four 8-byte stores. A 32-byte load cannot be store-forwarded from four 8-byte stores; it waits on L1, roughly ten cycles. Most of this type loads a UInt256 as a single Vector256 (AddAvx2, LessThanAvx2, ToBigEndian), as does the EVM stack. Measured against 392d749, that path was 1.53-1.58x slower than base for a caller reading the result as 32 bytes, while looking 0.81-0.91x (faster) for a caller reading .u0/.u3. A benchmark that consumes the result narrowly cannot see it.

That is consistent with this PR's own earlier finding of +1.3% on the eth_call replay alongside a large microbenchmark win.

main has since changed the four-argument constructor to write plain limbs (#103/#104), so this specific gap has narrowed: against 3a03caf the old implementation measures 0.86-0.96 across the same four shapes, with no regression. The store-width effect itself did not go away, it became main's behaviour, and 25 sites in UInt256.cs still read a UInt256 as a Vector256. SetLimbs therefore stores through Unsafe.As rather than returning a UInt256 (struct promotion splits the latter back into limb stores), and falls back to limb stores where Vector256 is not hardware accelerated, because there the callers read limbs too and the software Vector256.Create is out-of-line calls.

Measurements

Headline: this saves between roughly half a cycle and roughly twelve cycles per shift call, depending on how the caller reads the result. The large, reliable win is that callers which load the result as one 32-byte value stop paying a store-to-load forwarding stall. Everything else is a modest tightening worth one to four cycles.

Zen 5 (9950X), .NET 10, Tier-1 + Dynamic PGO, baseline main at 3a03caf, on a Solidity-idiomatic shift mix (selector/address/byte extraction, power-of-two scaling, plus out-of-range counts).

caller shape	base	this PR	saved / call	ratio
reads 4x8B, throughput - Rsh	2.254 ns	1.810 ns	0.44 ns (~1.9 cyc)	0.805
reads 4x8B, throughput - Lsh	2.298 ns	1.893 ns	0.41 ns (~1.7 cyc)	0.832
reads 1x32B, throughput - Rsh	4.509 ns	1.938 ns	2.57 ns (~11.1 cyc)	0.440
reads 1x32B, throughput - Lsh	4.673 ns	1.813 ns	2.86 ns (~12.3 cyc)	0.392
reads 4x8B, dependency chain - Rsh	2.938 ns	2.811 ns	0.13 ns (~0.5 cyc)	0.974
reads 4x8B, dependency chain - Lsh	2.999 ns	2.764 ns	0.24 ns (~1.0 cyc)	0.911
reads 1x32B, dependency chain - Rsh	5.647 ns	4.613 ns	1.03 ns (~4.4 cyc)	0.825
reads 1x32B, dependency chain - Lsh	5.718 ns	4.709 ns	1.01 ns (~4.3 cyc)	0.823

Cycle figures assume 4.3 GHz. Nothing regresses in any shape.

What the shapes mean. "Reads 1x32B" is a caller that loads the shift result as a single Vector256 - AddAvx2, SubtractAvx2, LessThanAvx2, ToBigEndian, and the EVM stack. "Reads 4x8B" is a caller that touches individual limbs. "Throughput" is independent shifts back to back; "dependency chain" feeds each result into the next shift, so it measures latency. Real code sits between the two, and the 32-byte row is the common one for this type.

Where the 11-12 cycles come from. A 32-byte load cannot be store-forwarded from four 8-byte stores; it waits on L1. main writes the result as limb stores, so a Vector256-reading caller stalls. Writing one 32-byte store removes the stall, which is why that row is the outlier and why the two consumer widths are reported separately.

Method and caveats

Both variants live in one process; each round measures baseline and candidate with rotating order; per-round ratios are reduced to a median, and that is repeated across 9 independent processes with the median taken again. An A/A control (baseline in the candidate slot) sits at 0.999-1.000 in every cell, min 0.994, max 1.005.

• Absolute times include an out-of-line call in both arms, identical on each side. Deltas are therefore the meaningful figure and the ratios understate the change to the shift body itself.
• The 32-byte cells are sensitive to code layout, which shifts store-to-load forwarding. Medians reproduced closely across two separate 9-process aggregations (0.440/0.392 and 0.440/0.394 for throughput), but individual processes have shown as much as 0.937 on one and 1.251 on another. Do not draw conclusions about these two rows from a single process, in either direction.
• This is one machine and one synthetic shift distribution. It does not tell you how often each caller shape occurs in a real block.

Code size

JitAsm, Tier-1 + PGO, x64:

	main @ 3a03caf	this PR
Lsh	338 B	367 B
Rsh	337 B	373 B
Lsh, DOTNET_EnableHWIntrinsic=0	377 B	335 B
Rsh, DOTNET_EnableHWIntrinsic=0	376 B	317 B

Neither body emits a jump table or an indirect jump. Under FullOpts main emits two out-of-line Lsh192 calls and this emits none. For reference, the superseded 8baac47 body was 510/512 B at Tier-1.

Correctness

• Full suite: 576,401 passed / 0 failed with intrinsics on, and 576,400 passed / 1 skipped with DOTNET_EnableHWIntrinsic=0 (the skip is the pre-existing hardware-accelerated hash test, unrelated to shifts).
• Focused shift suite: 467 tests, 0 skipped, also verified under DOTNET_EnableAVX2=0 and DOTNET_EnableAVX512F=0.
• CI matrix is green on Windows, Linux, macOS and ubuntu-24.04-arm, in HWIntrinsics and NoHWIntrinsics, debug and release, plus the zkEVM variant. ARM64 exercises the limb-store branch of SetLimbs, where Vector256.IsHardwareAccelerated is false.
• Cross-checked out of tree against a BigInteger oracle: every count in [-300, 300] plus int.MinValue, int.MaxValue and +/-2^20, over 72 values, for Lsh, Rsh and both aliased forms.

The test file no longer gates negative non-word counts behind X86Base.X64.IsSupported; with one implementation there is nothing left to skip, and it now covers every count in [-260, -1] on every target. Negative counts are pinned rather than designed: the pre-1.6.1 path produced those results by violating its own Debug.Assert(n < 64), so the behaviour is release-build fallout, not a contract. Pinning it means it can be retired deliberately instead of by accident, and defining n as unsigned would be a reasonable follow-up.

Outstanding

• The seeded 497-record eth_call replay needs re-running against 71be7f6. The result recorded below (+1.3% median, "held for rework") measured 8baac47, which is no longer the implementation, and the baseline it used has also moved.
• Also measured and rejected: an AVX2 branchless word shift (vpermd with a sign-masked index, 133 B) and an AVX512VL vpermt2q variant. The AVX2 form wins the throughput shapes 0.60-0.75x but loses the dependency-chain shapes, and averaged over 24 points the portable scalar funnel matched the AVX512 version, so no ISA gate was added. Three ways of sharing the funnel across the word-offset arms (goto case fall-through, and two switch-selection forms) are 30-40% smaller and all slower, because the duplication is specialisation: wordShift == 3 needs one shl and sharing forces all four funnels on every call.

---

Superseded: original evidence for the X86Base.X64-gated implementation (8baac47)

Retained for history. All figures below describe 8baac47 against base 392d749; both the implementation and the baseline have since changed, so these numbers no longer describe this PR.

Original summary

• Keep the origin/main logical-shift implementation as the default on ARM64 and when x64 hardware intrinsics are disabled.
• Dispatch to a direct-limb implementation only when X86Base.X64.IsSupported.
• Preserve word-shift fast paths and existing negative-count, boundary, and aliasing semantics.

Hosted microbenchmark evidence

• Candidate: bench-run/shifts-candidate-x86only-cc8a702, source cc8a702, workflow head 3cc977ca393ba5abfc27d79fc5bea818cb7e8a30, run 33175200432
• Exact base: bench-run/shifts-base-x86only-c020944, production source 392d749, benchmark-harness head b3adc40d3885e43127a51f129db852e054a02d61, run 33175617930

CorpusWeighted used the captured 497-corpus rates: 88.54% non-word Lsh counts and 68.17% non-word Rsh counts.

Host / mode	Lsh exact-base -> candidate	Rsh exact-base -> candidate
AMD64 HW	8.759 -> 5.805 ns (-33.73%)	8.854 -> 6.240 ns (-29.52%)
AMD64 no-HW	5.072 -> 5.123 ns (+1.01%)	5.310 -> 5.308 ns (-0.04%)
ARM64 HW	4.175 -> 4.204 ns (+0.69%)	4.273 -> 4.267 ns (-0.14%)
ARM64 no-HW	4.171 -> 4.203 ns (+0.77%)	4.264 -> 4.280 ns (+0.38%)

AMD64 HW per-workload deltas:

• Lsh: NonWord -46.29%, Word64 -24.90%, Word128 -32.40%, Word192 -32.75%, Zero -40.89%, OutOfRange256 -51.84%, OutOfRange257 -57.12%.
• Rsh: NonWord -39.41%, Word64 -36.15%, Word128 -25.49%, Word192 -34.16%, Zero -41.39%, OutOfRange256 -51.97%, OutOfRange257 -52.30%.

Each of those workloads fixes the shift count, which is what made the jump table and the narrow result consumer look free.

Original JIT/code-size check

With tiering disabled and COMPlus_JitDisasm: x64 HW Lsh 825 bytes, Rsh 903 bytes; x64 no-HW Lsh/Rsh 461/495 bytes. Measured under FullOpts rather than Tier-1, and compared against a prior candidate rather than against origin/main.

Isolated target-corpus update (2026-08-29)

Staged as 1.6.1-alpha.24b against exact Nethermind master in the seeded AMD 497-record eth_call workflow: run 33247079804.

• CPU/request +1.4%; average +2.2%; median +2.3%; p95 +1.6%; p99 +1.9%
• paired 40-pass replay median +1.3% slower (95% CI +1.1% to +1.6%); replay throughput -1.5%
• parity 497/497; measured failures/drops 0/0; snapshot pristine

Conclusion at the time: held for rework, not recommended for shipping unchanged despite the x64 microbenchmark win. That regression is what the store-width analysis above explains.

[Copilot]

🟢 Approval recommended

Pull request overview

This PR rewrites UInt256 logical left/right shifts to use a single portable “funnel shift” implementation, eliminating the prior word-shift helper methods and any architecture-gated duplicate implementations. It also adds a dedicated unit test suite that validates boundary, aliasing, negative-count legacy behavior, and randomized correctness against a BigInteger oracle.

Changes:

• Reimplemented UInt256.Lsh and UInt256.Rsh using a unified funnel-shift approach with only a word-offset branch and a single full-width store via SetLimbs.
• Removed unused shift helpers (NativeLsh/NativeRsh, Lsh64/128/192, Rsh64/128/192) and verified there are no remaining references.
• Added UInt256ShiftTests to exercise shift semantics (including aliasing and negative-count legacy behavior) against a BigInteger oracle.

File summaries

File	Description
src/Nethermind.Int256/UInt256.cs	Replaces Lsh/Rsh with a portable funnel shift and introduces SetLimbs to store results in a single full-width write when Vector256 is hardware accelerated.
src/Nethermind.Int256.Tests/UInt256ShiftTests.cs	Adds comprehensive shift correctness tests (boundaries, aliasing, full range up to 256, negative legacy behavior, randomized oracle checks).

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.