Add ARM SVE support

Author: jtl06

CMakeLists.txt

@@ -45,6 +45,7 @@ set(LIBDIVIDE_SSE2  AUTO CACHE STRING  "Enable SSE2 vector instructions")
 set(LIBDIVIDE_AVX2  AUTO CACHE STRING  "Enable AVX2 vector instructions")
 set(LIBDIVIDE_AVX512 AUTO CACHE STRING "Enable AVX512 vector instructions")
 set(LIBDIVIDE_NEON AUTO CACHE STRING "Enable ARM NEON vector instructions")
+set(LIBDIVIDE_SVE  AUTO CACHE STRING  "Enable ARM SVE vector instructions")
 
 # By default enable release mode ###############################
 
@@ -75,7 +76,7 @@ if (WARN_VEC_CONVERSIONS)
 endif()
 
 
-# Check if x86/x64 CPU  ########################################
+# Check native CPU family ######################################
 
 # Note that check_cxx_source_runs() must not be used when
 # cross-compiling otherwise the following error will occur:
@@ -176,6 +177,23 @@ else()
     endif()
 endif()
 
+# SVE
+if (NOT LIBDIVIDE_SVE STREQUAL "AUTO")
+    set(LIBDIVIDE_SVE_ENABLED "${LIBDIVIDE_SVE}")
+else()
+    set(SVE_TEST "
+    #include <arm_sve.h>
+    int main()
+    {
+        return svcntd() == 0;
+    }")
+    if (CMAKE_CROSSCOMPILING)
+        check_cxx_source_compiles("${SVE_TEST}" LIBDIVIDE_SVE_ENABLED)
+    else()
+        check_cxx_source_runs("${SVE_TEST}" LIBDIVIDE_SVE_ENABLED)
+    endif()
+endif()
+
 # AVX512
 if (NOT LIBDIVIDE_AVX512 STREQUAL "AUTO")
     set(LIBDIVIDE_AVX512_ENABLED "${LIBDIVIDE_AVX512}")
@@ -226,6 +244,9 @@ set(LIBDIVIDE_VECTOR_EXT "")
 if(LIBDIVIDE_NEON_ENABLED)
     list(APPEND LIBDIVIDE_VECTOR_EXT "LIBDIVIDE_NEON")
 endif()
+if(LIBDIVIDE_SVE_ENABLED)
+    list(APPEND LIBDIVIDE_VECTOR_EXT "LIBDIVIDE_SVE")
+endif()
 if(LIBDIVIDE_AVX512_ENABLED)
     list(APPEND LIBDIVIDE_VECTOR_EXT "LIBDIVIDE_AVX512")
 endif()

README.md

@@ -5,7 +5,7 @@
 
 ```libdivide.h```  is a header-only C/C++ library for optimizing integer division. Integer division is one of the slowest instructions on most CPUs e.g. on current x64 CPUs a 64-bit integer division has a latency of up to 90 clock cycles whereas a multiplication has a latency of only 3 clock cycles. libdivide allows you to replace expensive integer division instructions by a sequence of shift, add and multiply instructions that will calculate the integer division much faster.
 
-On current CPUs you can get a **speedup of up to 10x** for 64-bit integer division and a speedup of up to to 5x for 32-bit integer division when using libdivide. libdivide also supports [SSE2](https://en.wikipedia.org/wiki/SSE2), [AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) and [AVX512](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) vector division which provides an even larger speedup. You can test how much speedup you can achieve on your CPU using the [benchmark](#benchmark-program) program.
+On current CPUs you can get a **speedup of up to 10x** for 64-bit integer division and a speedup of up to 5x for 32-bit integer division when using libdivide. libdivide also supports [SSE2](https://en.wikipedia.org/wiki/SSE2), [AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions), [AVX512](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions), NEON, and SVE vector division, which can provide an even larger speedup. You can test how much speedup you can achieve on your CPU using the [benchmark](#benchmark-program) program.
 
 libdivide is compatible with 8-bit microcontrollers, such as the AVR series: [the CI build includes a AtMega2560 target](test/avr/readme.md). Since low end hardware such as this often do not include a hardware divider, libdivide is particularly useful. In addition to the runtime [C](doc/C-API.md) & [C++](doc/CPP-API.md) APIs, a set of [predefined macros](constant_fast_div.h) and [templates](constant_fast_div.hpp) is included to speed up division by 16-bit constants: division by a 16-bit constant is [not optimized by avr-gcc on 8-bit systems](https://stackoverflow.com/questions/47994933/why-doesnt-gcc-or-clang-on-arm-use-division-by-invariant-integers-using-multip). 
 
@@ -106,9 +106,9 @@ Caveats of branchfree divider:
 ## Vector division
 
 libdivide supports [SSE2](https://en.wikipedia.org/wiki/SSE2),
-[AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) and
-[AVX512](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions)
-vector division on x86 and x64 CPUs. In the example below we divide the packed 32-bit integers inside an AVX512 vector using libdivide. libdivide supports 32-bit and 64-bit vector division for both signed and unsigned integers.
+[AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions),
+[AVX512](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions),
+NEON, and SVE vector division. In the example below we divide the packed 32-bit integers inside an AVX512 vector using libdivide. libdivide supports 16-bit, 32-bit, and 64-bit vector division for both signed and unsigned integers.
 
 ```C++
 #include "libdivide.h"
@@ -129,6 +129,7 @@ Note that you need to define one of macros below to enable vector division:
 * ```LIBDIVIDE_AVX2```
 * ```LIBDIVIDE_AVX512```
 * ```LIBDIVIDE_NEON```
+* ```LIBDIVIDE_SVE```
 
 ## Performance Tips
 
@@ -138,9 +139,9 @@ Note that you need to define one of macros below to enable vector division:
   choose the one that performs best. The branchfree divider is more likely to get auto
   vectorized by the compiler (if you compile with e.g. ```-march=native```). But don't forget
   that the unsigned branchfree divider cannot be 1.
-* Vector division is much faster for 32-bit than for 64-bit. This is because there are
-  currently no vector multiplication instructions on x86 to efficiently calculate
-  64-bit * 64-bit to 128-bit. 
+* On x86, vector division is much faster for 32-bit than for 64-bit. This is because
+  there are currently no vector multiplication instructions on x86 to efficiently
+  calculate 64-bit * 64-bit to 128-bit.
 
 ## Build instructions
 

doc/C-API.md

@@ -62,6 +62,29 @@ int64x2_t libdivide_s64_branchfree_do_vec128(int64x2_t numers, const struct libd
 
 You need to define ```LIBDIVIDE_NEON``` to enable NEON vector division.
 
+## libdivide SVE vector division
+
+```C
+/* libdivide SVE division */
+svuint16_t libdivide_u16_do_sve(svuint16_t numers, const struct libdivide_u16_t *denom);
+svint16_t libdivide_s16_do_sve(svint16_t numers, const struct libdivide_s16_t *denom);
+svuint32_t libdivide_u32_do_sve(svuint32_t numers, const struct libdivide_u32_t *denom);
+svint32_t libdivide_s32_do_sve(svint32_t numers, const struct libdivide_s32_t *denom);
+svuint64_t libdivide_u64_do_sve(svuint64_t numers, const struct libdivide_u64_t *denom);
+svint64_t libdivide_s64_do_sve(svint64_t numers, const struct libdivide_s64_t *denom);
+
+/* libdivide SVE branchfree division */
+svuint16_t libdivide_u16_branchfree_do_sve(svuint16_t numers, const struct libdivide_u16_branchfree_t *denom);
+svint16_t libdivide_s16_branchfree_do_sve(svint16_t numers, const struct libdivide_s16_branchfree_t *denom);
+svuint32_t libdivide_u32_branchfree_do_sve(svuint32_t numers, const struct libdivide_u32_branchfree_t *denom);
+svint32_t libdivide_s32_branchfree_do_sve(svint32_t numers, const struct libdivide_s32_branchfree_t *denom);
+svuint64_t libdivide_u64_branchfree_do_sve(svuint64_t numers, const struct libdivide_u64_branchfree_t *denom);
+svint64_t libdivide_s64_branchfree_do_sve(svint64_t numers, const struct libdivide_s64_branchfree_t *denom);
+```
+
+You need to define ```LIBDIVIDE_SVE``` and compile for a target with Arm SVE
+support to enable SVE vector division.
+
 ## libdivide SSE2 vector division
 
 ```C

doc/CPP-API.md

@@ -77,8 +77,53 @@ template <Branching ALGO>
 int64x2_t operator/=(int64x2_t &n, const divider<int64_t, ALGO> &div)
 ```
 
-You need to define ```LIBDIVIDE_NEON``` to enable SSE2 vector division.
+You need to define ```LIBDIVIDE_NEON``` to enable NEON vector division.
 
+## SVE vector division
+
+```C++
+// Overload of operator /
+template <Branching ALGO>
+svuint16_t operator/(svuint16_t n, const divider<uint16_t, ALGO> &div)
+
+template <Branching ALGO>
+svint16_t operator/(svint16_t n, const divider<int16_t, ALGO> &div)
+
+template <Branching ALGO>
+svuint32_t operator/(svuint32_t n, const divider<uint32_t, ALGO> &div)
+
+template <Branching ALGO>
+svint32_t operator/(svint32_t n, const divider<int32_t, ALGO> &div)
+
+template <Branching ALGO>
+svuint64_t operator/(svuint64_t n, const divider<uint64_t, ALGO> &div)
+
+template <Branching ALGO>
+svint64_t operator/(svint64_t n, const divider<int64_t, ALGO> &div)
+
+
+// Overload of operator /=
+template <Branching ALGO>
+svuint16_t operator/=(svuint16_t &n, const divider<uint16_t, ALGO> &div)
+
+template <Branching ALGO>
+svint16_t operator/=(svint16_t &n, const divider<int16_t, ALGO> &div)
+
+template <Branching ALGO>
+svuint32_t operator/=(svuint32_t &n, const divider<uint32_t, ALGO> &div)
+
+template <Branching ALGO>
+svint32_t operator/=(svint32_t &n, const divider<int32_t, ALGO> &div)
+
+template <Branching ALGO>
+svuint64_t operator/=(svuint64_t &n, const divider<uint64_t, ALGO> &div)
+
+template <Branching ALGO>
+svint64_t operator/=(svint64_t &n, const divider<int64_t, ALGO> &div)
+```
+
+You need to define ```LIBDIVIDE_SVE``` and compile for a target with Arm SVE
+support to enable SVE vector division.
 
 ## SSE2 vector division