Skip to content

Epic: Cross-platform native-Fortran + CPU-scaling + CUDA/decomposition benchmark - #94

Merged
neuromechanist merged 5 commits into
mainfrom
feature/issue-84-epic-xplat-benchmark
Jul 9, 2026
Merged

Epic: Cross-platform native-Fortran + CPU-scaling + CUDA/decomposition benchmark#94
neuromechanist merged 5 commits into
mainfrom
feature/issue-84-epic-xplat-benchmark

Conversation

@neuromechanist

Copy link
Copy Markdown
Member

Summary

Cross-platform result + performance benchmark of every AMICA backend, extending the Phase B
dimension-sweep harness (#77) with native Fortran, CPU core-scaling, and full-decomposition
cross-backend equivalence. Answers how the native reference and multi-core CPU compare to the
GPU backends, where the crossovers are, and -- the headline -- whether every backend recovers
the same decomposition.

Closes #84

Phases (each individually reviewed + merged into this epic branch)

  1. Native-Fortran adapter + host setup (Phase 1: Native-Fortran adapter + CUDA host setup #85, PR Phase 1: Native-Fortran adapter + CUDA host setup #88): native-fortran-f64 backend in the
    dimsweep harness, timed via amica's own per-iteration stamps; a cross-platform gfortran+OpenMP
    build (benchmarks/fortran/) that compiles on macOS arm64 and Linux x86_64 without MKL. The
    build-portability fixes were also upstreamed to sccn/amica (PR Return best iterate from AMICATorchNG.fit (#51) #53, closing their Implement save/load persistence for AMICATorchNG #44).
  2. CPU core-count scaling (Phase 2: CPU core-count scaling sweep #86, PR Phase 2: CPU core-count scaling sweep #89): --threads sweep (torch-cpu / numpy / native-fortran)
    x channels x f64/f32 on both machines. Native Fortran is the only CPU backend that scales with
    cores and beats the RTX 4090 f64; torch-cpu peaks ~8 cores then regresses; MLX is the
    efficiency winner
    (~33 ms/it, matches a 4090 and a 32-core CPU on a laptop GPU); f32 verified
    correct (LL matches f64).
  3. Full-decomposition cross-backend equivalence (Phase 3: Full matrix + CUDA multi-model cross-platform report #87, PR Phase 3: full-decomposition cross-backend equivalence #93): every backend to 2000 iters,
    Hungarian-matched ICs + EEGLAB-style variance-ordered, de-sphered IC topomaps. Every
    torch/MLX backend recovers an identical decomposition
    (f32==f64==CPU==CUDA==MPS==MLX at
    corr 1.000); native Fortran is equivalent where the problem is well-determined (0.997 at
    k=574), differing only on weakly-determined k=30 components (a clock-seed local optimum).
    Equivalence tracks the data-adequacy k factor.

Result

pyAMICA is a device- and precision-invariant, EEGLAB-AMICA-equivalent decomposition, with MLX
the standout for efficiency on Apple hardware and native Fortran the fastest CPU reference.
Findings + figures in .context/issue-84/phase{1,2,3}_*.

Test plan

  • Each phase PR CI-green and reviewed (pr-review-toolkit, Sonnet); all findings addressed.
  • Real EEG only (ds002718), NO MOCKS. Benchmarks are hardware-specific (not in CI); unit tests
    for the adapters/matchers run in CI.

Follow-ups filed

#90 data-size/k sweep (full 747k recording), #91 spatially-distributed channel subsets,
#92 drop-in EEGLAB output/convention parity.

* Add native-fortran backend to dimsweep harness

* Make native amica build portable (gfortran, no MKL)

* Document Phase 1 build + cross-platform validation

* Address PR review: guard coverage, loud failures, doc fixes
* Add CPU core-count scaling sweep (--threads)

* Suppress AMICA_NumPy verbose output in sweep

* Add Phase 2 CPU-scaling findings

* Address review: numpy divergence, thread leak, report visibility

* Add complete f64+f32 cross-platform scaling findings
* Add Phase 3 decomposition-equivalence harness

* Add IC topomap figure + montage to decompose harness

* Order IC topomaps by variance (EEGLAB); square equivalence matrix

* Add Phase 3 decomposition-equivalence findings + figures

* Organize Phase 3 figures under benchmarks/figures/

* Address review: de-sphere topomap scalp maps, n<2 guard, doc fixes
@neuromechanist
neuromechanist merged commit e5c58ee into main Jul 9, 2026
5 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-84-epic-xplat-benchmark branch July 9, 2026 02:59
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.44%. Comparing base (9700752) to head (7fac897).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #94      +/-   ##
==========================================
+ Coverage   79.03%   79.44%   +0.40%     
==========================================
  Files          14       14              
  Lines        1975     1975              
  Branches      337      337              
==========================================
+ Hits         1561     1569       +8     
+ Misses        310      301       -9     
- Partials      104      105       +1     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epic: Cross-platform native-Fortran + CPU-scaling + CUDA benchmark

1 participant