Skip to content

Add MLX compatibility for IncrementalCCA #25

Description

@cboulay

Problem

IncrementalCCA describes its implementation as Array API compatible, but it does not currently run with MLX arrays.

Reproduced on an M4 Pro with MLX 0.32.0:

import mlx.core as mx
from ezmsg.learn.model.cca import IncrementalCCA

model = IncrementalCCA(n_components=1)
model.partial_fit(mx.random.normal((32, 4)), mx.random.normal((32, 3)))

This fails immediately with:

AttributeError: module 'mlx.core.linalg' has no attribute 'matrix_transpose'

Additional compatibility blockers

  • mlx.core.linalg does not provide matrix_transpose; use a namespace-level transpose/permutation operation instead.
  • mlx.core.linalg does not provide matrix_norm; the Frobenius norm can be expressed with elementwise operations and a reduction.
  • eigh and svd require an explicit MLX CPU stream. Unified memory means their results can remain MLX arrays.
  • initialize currently requests float64 on the input device, but MLX GPU arrays do not support float64.
  • bool(xp.any(...)) and float(...) in the adaptive-smoothing path introduce per-update device-to-host synchronization.

Affected code is in src/ezmsg/learn/model/cca.py.

Proposed scope

  • Preserve MLX input/state/output arrays and use an MLX-supported floating dtype.
  • Replace unsupported linalg transpose and norm calls with portable Array API operations.
  • Schedule MLX eigendecomposition and SVD explicitly on mx.cpu.
  • Review the adaptive-smoothing scalar state so unavoidable synchronization is documented and avoidable synchronization is removed.
  • Add MLX tests comparing partial_fit and transform against NumPy across multiple updates.
  • Profile the resulting implementation; the decompositions are CPU-only, so correctness should be established before claiming a performance benefit.

Acceptance criteria

  • IncrementalCCA.partial_fit and transform complete with MLX float32 inputs.
  • Learned matrices, weights, correlations, and transformed outputs remain MLX arrays.
  • Numerical results match the NumPy implementation within an appropriate float32 tolerance.
  • Tests cover first and subsequent adaptive updates and update_projections=True/False.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions