Skip to content

Use rank-2 linear for UMA spectral gates - #2141

Open
mlazos wants to merge 1 commit into
mainfrom
uma-rank2-spectral-gates
Open

Use rank-2 linear for UMA spectral gates#2141
mlazos wants to merge 1 commit into
mainfrom
uma-rank2-spectral-gates

Conversation

@mlazos

@mlazos mlazos commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

SpectralAtomwise retains a singleton coefficient dimension when projecting
its scalar slice. That sends [num_atoms, 1, sphere_channels] tensors through
Linear, which lowers the production shape to broadcast BMMs in the forward
and input-gradient graphs.

Squeeze the known singleton dimension before the scalar MLP. The gate already
accepts the resulting rank-2 tensor, so model semantics are unchanged while
Inductor can emit ordinary addmm/mm calls without layout conversions.

Performance

With torch.compile(dynamic=True), rank-2 improved the full endpoint from 16.98888 to 16.81807 ms with fresh caches, a 0.17081 ms (1.01%) gain, and from 16.99091 to 16.83604 ms with warm caches, a 0.15487 ms (0.91%) gain. It removed eight symbolic broadcast BMM calls.

The isolated scalar-MLP forward and input-VJP benchmark improved from 0.36388 to 0.35275 ms, a 3.1% gain.

Configuration: H100, TF32, UMA-S-1p2, 1,000 atoms, energy/forces/stress, umas_fast_gpu, merge_mole=True, external_graph_gen=False, internal graph v3, internal_graph_skin=0, compile=True, compile_dynamic_shapes=True, and full CUDA graph replay. Steady-state execution had zero recompiles and zero dynamic graph fallbacks.

Testing

PYTHONPATH=$PWD/src:/data/users/mlazos/pytorch python -m pytest -q tests/core/models/uma/test_escn_md.py
ruff check --ignore UP035 src/fairchem/core/models/uma/escn_md_block.py tests/core/models/uma/test_escn_md.py
git diff --check

All 13 tests passed.

Authored with assistance from Codex.

SpectralAtomwise kept a singleton coefficient dimension when projecting its scalar slice. Linear consequently lowered the production [1000, 1, 128] input to four broadcast BMMs in both the forward and input-gradient graphs.

Squeeze that known singleton dimension before the scalar MLP. GateActivation already consumes the resulting rank-2 tensor, so this changes no model semantics and lets Inductor emit ordinary addmm/mm calls without layout conversions. The focused test compares the old expression with the new path bitwise on CPU and records the Linear input rank.

The exact internal-v3, external_graph_gen=False endpoint improved by an order-balanced 0.250852 ms (1.353%), with all six paired cycles positive and all seven energy/forces/stress perturbations passing.

Test Plan:
```
PYTHONPATH=$PWD/src:/data/users/mlazos/pytorch python -m pytest -q tests/core/models/uma/test_escn_md.py
ruff check --ignore UP035 src/fairchem/core/models/uma/escn_md_block.py tests/core/models/uma/test_escn_md.py
git diff --check
```

Authored with assistance from Codex.
@meta-cla meta-cla Bot added the cla signed label Aug 6, 2026
@mlazos
mlazos marked this pull request as ready for review August 6, 2026 21:40
@mlazos
mlazos requested a review from rayg1234 August 6, 2026 21:40
@mlazos mlazos added the patch Patch version release label Aug 6, 2026
@lbluque lbluque added the enhancement New feature or request label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed enhancement New feature or request patch Patch version release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants