Skip to content

Commit c2d5bca

Browse files
committed
Use rank-2 linear for UMA spectral gates
SpectralAtomwise kept a singleton coefficient dimension when projecting its scalar slice. Linear consequently lowered the production [1000, 1, 128] input to four broadcast BMMs in both the forward and input-gradient graphs. Squeeze that known singleton dimension before the scalar MLP. GateActivation already consumes the resulting rank-2 tensor, so this changes no model semantics and lets Inductor emit ordinary addmm/mm calls without layout conversions. The focused test compares the old expression with the new path bitwise on CPU and records the Linear input rank. The exact internal-v3, external_graph_gen=False endpoint improved by an order-balanced 0.250852 ms (1.353%), with all six paired cycles positive and all seven energy/forces/stress perturbations passing. Test Plan: ``` PYTHONPATH=$PWD/src:/data/users/mlazos/pytorch python -m pytest -q tests/core/models/uma/test_escn_md.py ruff check --ignore UP035 src/fairchem/core/models/uma/escn_md_block.py tests/core/models/uma/test_escn_md.py git diff --check ``` Authored with assistance from Codex.
1 parent cb4f933 commit c2d5bca

2 files changed

Lines changed: 42 additions & 1 deletion

File tree

src/fairchem/core/models/uma/escn_md_block.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -321,7 +321,8 @@ def __init__(
321321
)
322322

323323
def forward(self, x):
324-
gating_scalars = self.scalar_mlp(x.narrow(1, 0, 1))
324+
scalar_input = x.narrow(1, 0, 1).squeeze(1)
325+
gating_scalars = self.scalar_mlp(scalar_input)
325326
x = self.so3_linear_1(x)
326327
x = self.act(gating_scalars, x)
327328
x = self.so3_linear_2(x)

tests/core/models/uma/test_escn_md.py

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,46 @@
1818

1919
from fairchem.core.datasets.atomic_data import AtomicData
2020
from fairchem.core.models.uma.escn_md import eSCNMDBackbone, resolve_dataset_mapping
21+
from fairchem.core.models.uma.escn_md_block import SpectralAtomwise
22+
23+
24+
@pytest.mark.parametrize(
25+
"num_atoms, sphere_channels, hidden_channels, lmax",
26+
[(1, 8, 12, 1), (7, 16, 20, 2), (7, 128, 256, 2)],
27+
)
28+
def test_spectral_atomwise_scalar_projection_is_rank_two(
29+
num_atoms: int, sphere_channels: int, hidden_channels: int, lmax: int
30+
) -> None:
31+
torch.manual_seed(0)
32+
module = SpectralAtomwise(
33+
sphere_channels=sphere_channels,
34+
hidden_channels=hidden_channels,
35+
lmax=lmax,
36+
mmax=lmax,
37+
SO3_grid=None,
38+
)
39+
baseline_input = torch.randn(
40+
num_atoms, (lmax + 1) ** 2, sphere_channels, requires_grad=True
41+
)
42+
candidate_input = baseline_input.detach().clone().requires_grad_(True)
43+
44+
gating_scalars = module.scalar_mlp(baseline_input.narrow(1, 0, 1))
45+
baseline = module.so3_linear_1(baseline_input)
46+
baseline = module.act(gating_scalars, baseline)
47+
baseline = module.so3_linear_2(baseline)
48+
baseline_grad = torch.autograd.grad(baseline.sum(), baseline_input)[0]
49+
50+
linear_input_shapes = []
51+
hook = module.scalar_mlp[0].register_forward_pre_hook(
52+
lambda _module, args: linear_input_shapes.append(args[0].shape)
53+
)
54+
candidate = module(candidate_input)
55+
hook.remove()
56+
candidate_grad = torch.autograd.grad(candidate.sum(), candidate_input)[0]
57+
58+
assert linear_input_shapes == [(num_atoms, sphere_channels)]
59+
torch.testing.assert_close(candidate, baseline, rtol=0, atol=0)
60+
torch.testing.assert_close(candidate_grad, baseline_grad, rtol=0, atol=0)
2161

2262

2363
@pytest.mark.parametrize(

0 commit comments

Comments
 (0)