ScalarQuantizer: split SIMD specializations into per-SIMD TUs + DD dispatch (#4839)

Summary:
Pull Request resolved: #4839

Split the SIMD-gated template specializations out of ScalarQuantizer.cpp
and the shared headers into per-SIMD compilation units and wire up the
Dynamic Dispatch (DD) infrastructure (COMPILE_SIMD_*, with_simd_level).

**What moved where**

- SIMD specializations removed from `codecs.h`, `quantizers.h`,
  `similarities.h`, `distance_computers.h` — these now contain only
  primary templates and scalar (`SIMDLevel::NONE`) specializations.
  (Most use empty primary templates; `quantizers.h` uses an inheriting
  fallback pattern for `QuantizerFP16`, `QuantizerBF16`, etc.)
- SIMD specializations moved into `sq-avx2.cpp` / `sq-avx512.cpp` /
  `sq-neon.cpp`, each guarded by `COMPILE_SIMD_*`.
- `sq-generic.cpp` deleted — the `NONE` level is now instantiated
  directly in `ScalarQuantizer.cpp` via `sq-dispatch.h`.
- `sq-inl.h` renamed to `scanners.h`.

**Dispatch mechanism**

- `sq-dispatch.h` is an X-macro-style header: each per-SIMD `.cpp` file
  `#define`s `THE_LEVEL_TO_DISPATCH` and `#include`s it to stamp out
  explicit template specializations of the selection functions
  (`sq_select_quantizer`, `sq_select_distance_computer`,
  `sq_select_InvertedListScanner`).
- `ScalarQuantizer.cpp` uses `with_simd_level` for runtime dispatch
  and instantiates the `NONE` level via the same `sq-dispatch.h`.
- Each per-SIMD selection function returns `nullptr` when the dimension
  doesn't align, and the caller falls back to `NONE`.
- `sq-neon.cpp` handles both `ARM_NEON` and `ARM_SVE` (SVE forwards
  to NEON — no dedicated SVE SQ implementation yet).

**Build**

- `xplat.bzl`, `CMakeLists.txt` — register new SIMD source files and
  headers.
- Within the SQ module, `COMPILE_SIMD_*` macros gate all SIMD code
  paths. (Compiler-defined macros like `__AVX2__` are still used in
  lower-level shared headers like `simdlib.h` and `fp16.h`.)

Reviewed By: mdouze

Differential Revision: D94375408

fbshipit-source-id: a07c31540242defcc605dd74e07bd25b8c163f43

Author: algoriddle

faiss/CMakeLists.txt

@@ -10,13 +10,16 @@
 # =============================================================================
 set(FAISS_SIMD_AVX2_SRC
   impl/pq_code_distance/pq_code_distance-avx2.cpp
+  impl/scalar_quantizer/sq-avx2.cpp
   utils/simd_impl/distances_avx2.cpp
 )
 set(FAISS_SIMD_AVX512_SRC
   impl/pq_code_distance/pq_code_distance-avx512.cpp
+  impl/scalar_quantizer/sq-avx512.cpp
   utils/simd_impl/distances_avx512.cpp
 )
 set(FAISS_SIMD_NEON_SRC
+  impl/scalar_quantizer/sq-neon.cpp
   utils/simd_impl/distances_aarch64.cpp
 )
 set(FAISS_SIMD_SVE_SRC
@@ -246,7 +249,9 @@ set(FAISS_HEADERS
   impl/scalar_quantizer/codecs.h
   impl/scalar_quantizer/distance_computers.h
   impl/scalar_quantizer/quantizers.h
+  impl/scalar_quantizer/scanners.h
   impl/scalar_quantizer/similarities.h
+  impl/scalar_quantizer/sq-dispatch.h
   impl/scalar_quantizer/training.h
   impl/ThreadedIndex-inl.h
   impl/ThreadedIndex.h
@@ -440,7 +445,7 @@ if(FAISS_OPT_LEVEL STREQUAL "dd")
       set_source_files_properties(${FAISS_SIMD_AVX512_SRC}
         TARGET_DIRECTORY faiss
         PROPERTIES COMPILE_OPTIONS
-          "-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mpopcnt"
+          "-mavx512f;-mavx512cd;-mavx512vl;-mavx512dq;-mavx512bw;-mfma;-mf16c;-mpopcnt"
       )
     elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "(aarch64|arm64|ARM64)")
       # ARM NEON is always available on aarch64, no special compiler flags needed