Investigate remaining SIMD performance opportunities

[mrecachinas]

Context

The ARM64 optimization passes have exhausted the credible local opportunities: scalar and NEON kernels are close to their instruction floors, per-record dispatch is hoisted, the winning 16-byte specialization landed, Rayon/GIL crossovers are measured, and Python buffer overhead is minimized. Fixed-width 32-byte specialization was neutral, while 64/128-byte specializations regressed and were reverted.

Only two material performance avenues remain:

Opportunities

1. x86-specific SIMD, measured on real hardware

   • Benchmark AVX2 and AVX-512 implementations on representative x86 CPUs.
   • Evaluate VPSHUFB lookup/popcount strategies for AVX2 and VPOPCNTB/VPOPCNTDQ-capable AVX-512 paths.
   • Test batched threshold (_with_max) reductions and retune dispatch cutovers.
   • Keep separate results for AVX2-only and AVX-512-capable machines; do not infer wins from generated assembly alone.

2. Cross-record SIMD for array scans

   • Process multiple fixed-width records per vector rather than invoking one kernel per record.
   • Prioritize 16- and 32-byte records for first, best, and all; expand only when measurements justify it.
   • Preserve first-index ordering, best-distance tie behavior, threshold short-circuiting, and exact-match termination.
   • Measure random no-match, early/mid/late match, and parallel-crossover workloads.

Acceptance criteria

• Before/after results from at least three benchmark runs on the same machine.
• End-to-end Python array benchmarks as well as raw Rust Criterion results.
• No public API or semantic changes.
• Revert experiments that do not outperform the current kernels outside benchmark noise.

Everything else identified during the performance review was either landed or measured and rejected.

[mrecachinas]

Completed the remaining SIMD investigation with measured results on representative ARM64, AVX-512, and AVX2-only hardware.

Landed:

• Speed up ARM fixed-width array scans #54: ARM NEON 16/32-byte cross-record scanners.
• Speed up AVX-512 byte and fixed-width array scans #55: masked AVX-512 short-byte path and AVX-512 16/32-byte cross-record scanners.
• Add measured Python batch APIs #56: measured Python batch APIs to amortize FFI overhead.

AVX2-only evaluation used three-run medians on a GCP e2-standard-4, Intel Broadwell-class CPU (family 6 model 79), with AVX2 and no AVX-512. The following isolated experiments were measured and reverted:

• AVX2 16/32-byte cross-record scanners: Rust scenarios ranged from 1.9% faster to 3.4% slower, with end-to-end Python scans unchanged. This was benchmark noise rather than a material win.
• AVX2 residual-tail staircase (16-byte shuffle + 8-byte POPCNT): the targeted 127-byte case regressed 2.5%; stable guard sizes were within noise.
• Short-input native/SSE routing inside the AVX2 entrypoint: explicit AVX2 timings remained within roughly 1% because cross-kernel call overhead erased the isolated native-kernel advantage.
• Batched horizontal reduction for the AVX2 threshold residual: 127 bytes regressed 5.2% and 512 bytes regressed 12.6%, with no stable compensating improvement.

The retained AVX-512 work produced 2.50-5.72x Python fixed-width scan speedups and 32.9-193.9% improvements for short byte inputs. All AVX2 experiments that failed to outperform the current kernels outside noise were reverted, satisfying the issue acceptance criteria.