Skip to content

AVX2: eight-wide f32 BODFT forward/inverse combines, FMA intrinsics and SIMD tail scheduling improvements#73

Merged
falseywinchnet merged 2 commits into
mainfrom
fwysqa-main/optimize-performance-with-avx2/sse2
Jun 27, 2026
Merged

AVX2: eight-wide f32 BODFT forward/inverse combines, FMA intrinsics and SIMD tail scheduling improvements#73
falseywinchnet merged 2 commits into
mainfrom
fwysqa-main/optimize-performance-with-avx2/sse2

Conversation

@falseywinchnet

Copy link
Copy Markdown
Owner

Motivation

  • Improve BODFT performance by adding an AVX2 eight-position single-precision forward and inverse combine so f32 work uses 256-bit lanes instead of falling back to 128-bit paths.
  • Reduce inverse overhead and tail penalties by tightening SIMD tail flow and running 128-bit V2/V4F tails after wider AVX2/AVX-512 loops.
  • Reduce instruction count and live pressure by preferring accumulator-first FMA intrinsics where available and folding constant 0.5 factors into twiddle constants for rotations.
  • Document kernel dependency/frontier decisions to guide further register-pressure-sensitive optimizations.

Description

  • Added AVX2 f32 helpers and eight-wide combine routines: bodft_uzp8_ps, bodft_store8_ps, bodft_rev8_ps, bodft_combine_fwd_avx2_f32, and bodft_combine_inv_avx2_f32, and wired them into combine_fwd/combine_inv dispatch for float (sizeof(RT)==4).
  • Updated x86 SIMD macros in bruun_kernel.hpp to use _mm_fmadd_* / _mm_fnmadd_* when __FMA__ is defined (falling back to separate mul+add/sub otherwise), and updated 128-bit float/double codelets to shorten live ranges and move loads after rotation start where profitable.
  • Reordered several AVX2/128-bit loops so wide SIMD runs first and 128-bit tails run afterwards, and folded the 0.5 scale into twiddle constants in float inverse loops to remove redundant half multiplies.
  • Added documentation and notes: new kernel_sequentiality_flow.md, updates to inverse_acceleration_notes.txt, and BODFT AVX2 notes (bodft_avx2_notes.md) describing validation and observed performance changes.

Testing

  • Ran the full test suite with AVX2/FMA: make test BUILD_DIR=build-avx2 CXXWARNFLAGS='-Wall -Wextra -Wpedantic -Werror', and the CTest checks passed.
  • Ran the standalone BODFT benchmark: make build-avx2/examples/bodft_benchmark BUILD_DIR=build-avx2 && build-avx2/examples/bodft_benchmark, and observed measured improvements (example: double inverse ~748 us -> ~611 us at N=65536; f32 inverse ~1.13 ms -> ~0.78 ms at N=65536).
  • Verified assembly with make asm-check BUILD_DIR=build-avx2 and inspected build-avx2/src/bodft_avx2.s for AVX/FMA instructions.
  • Also built/checked an SSE2/no-AVX configuration to confirm the 128-bit fallback paths remain correct.

Codex Task

@falseywinchnet falseywinchnet merged commit cfb88b3 into main Jun 27, 2026
4 checks passed
@falseywinchnet falseywinchnet deleted the fwysqa-main/optimize-performance-with-avx2/sse2 branch June 27, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant