Scalable Vector Extension (SVE) support

[jserv]

Quote from Arm website:

Scalable Vector Extension (SVE) is a vector extension the A64 instruction set of the Armv8-A architecture. Armv9-A builds on SVE with the SVE2 extension. Unlike other SIMD architectures, SVE and SVE2 do not define the size of the vector registers, but constrains it to a range of possible values, from a minimum of 128 bits up to a maximum of 2048 in 128-bit wide units. Therefore, any CPU vendor can implement the extension by choosing the vector register size that better suits the workloads the CPU is targeting. The design of SVE and SVE2 guarantees that the same program can run on different implementations of the instruction set architecture without the need to recompile the code.

genarchbench comes with partial sse2sve header, which can be regarded as a starting point for SSE2NEON to support SVE.

[jserv]

High-level view: where SVE/SVE2 can help SSE2NEON

From an SSE2NEON perspective, SVE/SVE2 is most attractive for intrinsics that are currently expensive to emulate with NEON, especially when operations involve:

• Horizontal reductions (sum / min / max)
• Dot products and multiply–add patterns
• Sum of absolute differences (SAD) and absolute-diff–plus–accumulate
• Saturating arithmetic and widening/narrowing operations
• Certain shuffle / table-lookup patterns
• Higher-level string scanning or token search behavior

In contrast, intrinsics that already map 1:1 to simple NEON operations (add, sub, mul, bitwise ops) usually do not gain much from SVE unless the surrounding algorithm is redesigned to actually exploit scalable vector length.

1. Horizontal reductions and dot-product-like patterns

SSE often implements horizontal reductions and dot products with multiple instructions:

• Multiply, then add in stages
• Use shuffle and add to accumulate within a 128-bit lane

In NEON, SSE2NEON does essentially the same thing using:

• Pairwise add (vpadd*)
• Manual vget_*/vcombine_* plus scalar adds

SVE/SVE2 provides direct reduction operations, for example:

• Sum of all elements: svaddv_*
• Min/max of all elements: svminv_*, svmaxv_*

Even if the vector length is fixed at 128 bits to match __m128 semantics, a multi-instruction NEON sequence can often be collapsed into:

1. Load vector
2. Operate
3. Single reduction intrinsic

Representative SSE intrinsics / patterns

1. _mm_hadd_ps, _mm_hadd_pd
   These do horizontal add within the vector. Typical NEON emulation uses multiple vector ops + shuffles. With SVE, it is possible to:

• Represent the “intermediate” vector
• Use svaddv_f32 or svaddv_f64 with an all-true predicate to reduce

This is especially beneficial if user code is already doing manual horizontal sums after _mm_mul_ps etc. The SVE version can cut several NEON instructions.

2. _mm_hadd_epi16, _mm_hadd_epi32
   Same story for integer reductions. NEON normally uses pairwise add and lane extraction. SVE2 can do:

• svaddv_s16 or svaddv_s32 (or unsigned variants)
• Scalar extraction from the result

3. Dot-product patterns using _mm_mul_ps + _mm_add_ps. User code often writes:

__m128 a, b;
// element-wise multiply
__m128 prod = _mm_mul_ps(a, b);
// then chain of add + shuffle to get scalar dot product

In an SVE-based backend, the implementation could:

• Keep mul as svmul_f32
• Use svaddv_f32 for the scalar dot, instead of explicit lane-wise adds

This simplifies the emulation of “dot product style” SSE code, even when there is no single SSE intrinsic like _mm_dp_ps involved.

2. Dot-product intrinsics and multiply–add

SVE2 introduces dot-product instructions (sdot, udot, etc.) that can implement many “multiply and add” patterns much more compactly than NEON.

Intrinsic worth targeting:

4. _mm_madd_epi16. Semantics: multiply pairs of 16-bit integers, then add them to produce 32-bit results.

On NEON, this usually expands to:

• vmull_s16 or vmlal_s16 type sequences
• Plus extra adds and shuffles to match SSE’s layout

With SVE2’s dot-product instructions for 8/16-bit integer elements, it is feasible to:

• Map the “multiply two adjacent 16-bit elements and accumulate into 32-bit” behavior onto an SVE2 dot macro, or at least reduce the number of intermediate NEON ops
• Implement a shorter and more regular sequence that also scales better with wider SVE vector lengths

Even if adhering to 128-bit SVE vectors, the dot instructions can reduce instruction count significantly.

5. _mm_dp_ps (SSE4.1) : This intrinsic does a floating-point dot product with a configurable mask for which lanes participate and which lanes get the result. On NEON, SSE2NEON usually emulates it via:

• mul, add, shuffle
• Possibly a cascade of scalar adds

With SVE:

• “Participating lanes” can be treated via predicates.
• The core sum can be implemented with svaddv_f32.
• The mask semantics (which lanes get the result) are then reconstructed by broadcasting the scalar result where needed.

SSE’s masking behavior must still be replicated, but the actual accumulation becomes much cheaper.

3. Sum-of-absolute-differences (SAD) and absolute-diff reductions

SSE’s _mm_sad_epu8 is a classic: it computes the sum of absolute differences of bytes in two vectors, producing partial sums in 64-bit lanes.

On NEON, SSE2NEON typically uses:

1. vabdq_u8 (byte-wise absolute diff)
2. Widen / accumulate: vpaddlq_u8 → vpaddlq_u16 → maybe more adds
3. Lane extraction and scalar additions

Intrinsic worth targeting

6. _mm_sad_epu8 : SVE2 extends the integer and DSP-friendly instruction set to cover more absolute-diff–and–accumulate patterns. With SVE2, the implementation can:

• Do absolute difference via svabd_u8 or similar
• Use widening adds plus reductions to compute the final sum in fewer instructions
• Rely on svaddv_u16/svaddv_u32 to collapse the result quickly

The end result is shorter and likely faster than the NEON emulation, especially in inner loops doing SAD over many vectors (e.g., motion estimation, block matching, or generic image diff).

4. Saturating arithmetic and narrowing/packing intrinsics

SSE has many intrinsics that combine saturating arithmetic with narrowing. NEON can do saturating ops and narrowing, but the sequences can be verbose when matching SSE semantics exactly.

SVE2 explicitly aims to generalize NEON-style fixed-point and DSP operations into scalable form, including:

• Saturating add/sub
• Widening and narrowing
• High/long multiply–add variants

Intrinsics worth targeting

7. _mm_adds_epi8, _mm_adds_epu8
8. _mm_adds_epi16, _mm_adds_epu16

These are signed/unsigned saturating additions. NEON emulation is straightforward but may require multiple steps to preserve exact SSE corner-case behavior. SVE2 has saturating add/sub instructions that can:

• Replace multi-instruction NEON patterns with fewer SVE2 operations
• Simplify the logic around overflow handling

9. _mm_packs_epi16, _mm_packus_epi16

These “pack and narrow” intrinsics take 16-bit elements and pack them into 8-bit with signed or unsigned saturation. NEON typically uses:

• Narrow operations (vqmovn_*)
• Interleaving/combine steps to match the SSE ordering

SVE2 offers scalable narrowing with saturation, which can:

• Compress multiple NEON steps into fewer SVE instructions
• Make the implementation easier to maintain and extend to wider SVE vector lengths later

These intrinsics are common in pixel pipelines, color-space transforms, and fixed-point audio/video processing, so the payoff in real workloads can be noticeable.

5. Shuffle and table-lookup patterns

Some shuffle intrinsics are already nicely matched by NEON (_mm_shuffle_ps → vcombine + vget patterns). Others, especially per-byte shuffles, are expensive.

Intrinsic worth targeting

10. _mm_shuffle_epi8 (PSHUFB, SSSE3) : This is a byte-level LUT/permutation within 128-bit lanes. NEON emulation often uses:

• vtbl/vtbl2 operations
• Extra operations to handle the “zeroing out” behavior where the top bit of each index byte masks the result

SVE/SVE2 has scalable table-lookup and permutation instructions that can generalize this behavior. For SSE2NEON, the backend can:

• Keep the semantics limited to a 128-bit slice
• Use SVE’s table-lookup intrinsics to implement the shuffle with fewer instructions than the NEON path
• Leverage predicates to handle the per-byte “masking to zero” behavior

The benefit is greatest in codecs, crypto, or text processing that do heavy table-based byte shuffles.

6. Higher-level string scanning and token matching

SSE4.2 string intrinsics (_mm_cmpestri, _mm_cmpistri, etc.) are already hard to emulate “intrinsic-accurately” on NEON. SSE2NEON typically falls back to C loops or complex sequences.

SVE2 introduces instructions like SVMATCH for multi-token text search and parsing workloads. While they don’t provide a direct 1:1 mapping to SSE4.2 string intrinsics, they open the door for:

• Fast-path implementations of common patterns (e.g., scanning for delimiters, parsing JSON-like data)
• A “best-effort” acceleration path for SSE-style string compare helpers that do not rely on exact SSE4.2 flags/encodings

These are probably out of scope for strict SSE2NEON API parity, but interesting if optional helpers or a “fast but not bit-perfect” mode are ever added for certain workloads.

7. What is not worth rewriting just for SVE?

From a pure SSE2NEON standpoint, SVE/SVE2 is less compelling when:

1. The SSE intrinsic already maps to one or two NEON instructions (e.g., _mm_add_ps, _mm_sub_ps, _mm_and_si128, _mm_or_si128). A direct SVE rewrite does not buy much unless the algorithm is also changed to use scalable vectors.
2. The intrinsic relies on x86-specific behavior like _mm_movemask_ps / _mm_movemask_epi8. Arm has no single instruction that exactly mirrors those semantics, so SVE does not magically solve this; a multi-step path is still required to compress per-lane sign bits into a scalar mask.
3. The intrinsic is tightly tied to 128-bit lane boundaries in ways that don’t generalize naturally to large SVE vector lengths. For those, NEON’s fixed 128-bit model is often clearer and simpler to maintain.