Skip to content

Remove fused q==8 inverse codelets and use scheduled d3+d3+norm inverse path; simplify twiddle helpers#39

Merged
falseywinchnet merged 1 commit into
mainfrom
ybeb6g-main/assess-d4-fusion-impact-on-segment-runner
Jun 21, 2026
Merged

Remove fused q==8 inverse codelets and use scheduled d3+d3+norm inverse path; simplify twiddle helpers#39
falseywinchnet merged 1 commit into
mainfrom
ybeb6g-main/assess-d4-fusion-impact-on-segment-runner

Conversation

@falseywinchnet

Copy link
Copy Markdown
Owner

Motivation

  • Replace fragile, per-ISA fused q==8 inverse codelets with a single, predictable scheduled inverse runner to reduce code size, compiler pressure, and cross-ISA variability.
  • Make twiddle lookup codepaths clearer and less branchy in scalar/f32 helpers to avoid nested ternary expressions in hot paths.
  • Keep transform math and public APIs unchanged while exposing a linear, forward-like switch schedule for inverse execution to improve portability and predictability.

Description

  • Removed/disabled fused q==8 inverse codelets for float and double (previous codelet_d4_inv_f32, codelet_d4_inv, and wide-ISA fused variants) and replaced q==8 handling with explicit two depth-3 codelet calls plus the q==8 norm (d3 + d3 + norm) in all schedule and recursive call sites.
  • Rewrote scalar and float twiddle helpers s_twiddle(int) and sf_twiddle(int) to use explicit if branches returning 0 for m==0, C[1]/CF[1] for m==1, and C[m^1]/CF[m^1] otherwise.
  • Updated all inverse schedule dispatch sites (rec_inv_res, run_inv_residue_schedule, rec_inv_res for double, and forward scheduling spots) to invoke the two d3 codelets and then norm_q_* for q==8, and removed conditional BRUUN_LEVEL paths that used the fused variants.
  • Updated documentation/notes (inverse_acceleration_notes.txt) to record the decision to keep q==8 on the shared d3 + d3 + norm path, list completed changes, and note benchmarking/assembly inspection results.

Testing

  • Ran assembly inspection and microbenchmarks for the scheduled inverse on an Intel Xeon Platinum 8370C with -O3 -mavx2 -mfma and native AVX-512 builds, and observed the scheduled inverse running and benchmark output including a direct Inv_ns column. Tests succeeded.
  • Performed local build and CI-style compilation across narrow and wide targets (SSE2/NEON, AVX2, AVX-512) to ensure the removed fused codelets did not break builds, and all builds completed successfully.

Codex Task

@falseywinchnet falseywinchnet merged commit f716b2f into main Jun 21, 2026
4 checks passed
@falseywinchnet falseywinchnet deleted the ybeb6g-main/assess-d4-fusion-impact-on-segment-runner branch June 23, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant