Skip to content

potential/backend: centrally coerce forced-backend scalar leaks#987

Merged
jobovy merged 2 commits into
feat/backendsfrom
backend/central-scalar-coercion
Jun 20, 2026
Merged

potential/backend: centrally coerce forced-backend scalar leaks#987
jobovy merged 2 commits into
feat/backendsfrom
backend/central-scalar-coercion

Conversation

@jobovy

@jobovy jobovy commented Jun 20, 2026

Copy link
Copy Markdown
Owner

What

Burndown PR-3b — the central scalar-coercion fix, built on #983's galpy.backend._coerce. Under the forced-backend all-backend suite, plain numpy/Python scalars leak into strict torch ops (torch.cos(0.0) / torch.sqrt(numpy.float64) / numpy.ndarray*Tensor all raise; jax is permissive). coerce_coords already handles passed coordinates; this closes the three remaining scalar-leak channels — all guarded so numpy stays byte-identical:

  1. conversion.py potential_physical_input: also coerce the phi/t signature defaults (bound via inspect.signature) when omitted, strictly inside the xp is not numpy and _check_backend_compatible guard — so internally-derived scalars (Ferrers _pa+_omegab*t, DehnenBar _smooth(t), CosmphiDisk/HenonHeiles/Logarithmic sin/cos(phi), …) see backend tensors.
  2. backend/_coerce.py promote_scalars: use is_backend_array() (not hasattr(v,'ndim')) as the leave-it test, so a numpy.float64 (has .ndim but torch rejects) is promoted.
  3. backend/_namespaces.py asarray_on_device: translate a numpy dtype arg to the backend dtype (torch.asarray(dtype=numpy.float64) raises) — fixes SpiralArms.

Plus 19 potentials: move self._backend_compatible=True above self.normalize(...) in __init__ (so construction-time normalize()→Rforce(1.,0.) is coerced); and FerrersPotential axisymmetric phi-reset 0.0 → zeros_like_backend(xp,R).

Impact

Flips ~331 torch test_potential ledger entries to pass (forceAsDeriv / evaluateAndDerivs / 2ndDeriv / poisson / normalize / toVertical_toPlanar, across the potential families). The ledger entries themselves are pruned by the separate ledger-regen PR.

Byte-identity + review

numpy path byte-identical (the whole mechanism is gated on xp is not numpy): verified hash-matched Phi/forces/dens across the 19 reordered potentials + Ferrers + planar default-phi paths; 281/281 numpy parity; all 3325 backend unit tests pass. Independently adversarially reviewed across 4 angles — zero blockers: byte-identity, the inspect.signature default-injection correctness (every signature shape: positional/kwarg/omitted, double-decoration via @wraps, DF-skip, None defaults, no kwonly/posonly), the 19 __init__ reorders, and regression (no previously-passing test breaks; deferred cases honestly still ledgered).

Deferred (separate follow-ups)

  • AnySphericalPotential — scipy interior; test_backend_conventions pins it as the canonical unmigrated potential, so it needs a coordinated source+test migration.
  • AnyAxisymmetricRazorThinDiskPotential — scipy interior.
  • specialSpiralArmsPotential — a torch float64 FD-precision artifact (the crash is fixed; the force matches numpy to 3.5e-8).

🤖 Generated with Claude Code

Under the forced-backend all-backend suite, plain numpy/Python scalars leak into
strict torch ops (torch.cos(0.0) / torch.sqrt(numpy.float64) / numpy.ndarray*Tensor
all raise; jax is permissive). coerce_coords already handles passed coordinates;
this closes the three remaining scalar-leak channels, all guarded so numpy stays
byte-identical:

* conversion.py potential_physical_input: also coerce the phi/t SIGNATURE
  DEFAULTS (bound via inspect.signature) when omitted, strictly inside the
  `xp is not numpy and _check_backend_compatible` guard -- so internally-derived
  scalars (Ferrers tp=_pa+_omegab*t, DehnenBar _smooth(t), CosmphiDisk/HenonHeiles
  /Logarithmic sin/cos(phi), ...) see backend tensors. numpy path never rebinds.
* backend/_coerce.py promote_scalars: use is_backend_array() (not hasattr ndim)
  as the "leave it" test, so a numpy.float64 (has .ndim but torch rejects) is
  promoted.
* backend/_namespaces.py asarray_on_device: translate a numpy dtype arg to the
  backend dtype (torch.asarray(dtype=numpy.float64) raises) -- fixes SpiralArms.
* 19 potentials: move self._backend_compatible=True ABOVE self.normalize(...) in
  __init__ so construction-time normalize()->Rforce(1.,0.) is coerced.
* FerrersPotential: axisymmetric phi-reset 0.0 -> zeros_like_backend(xp, R).

Flips ~30+ sampled torch test_potential entries (forceAsDeriv/evaluateAndDerivs/
2ndDeriv/poisson/normalize/toVertical_toPlanar). numpy byte-identical (verified:
281/281 + bit-identical Phi/forces across MN/NFW/Ferrers/PSPwC/CosmphiDisk/Log);
all backend unit tests pass (3325). Deferred (separate work): AnySphericalPotential
(scipy interior, pinned unmigrated by test_backend_conventions), AnyAxisymmetric
RazorThin (scipy), SpiralArms FD-precision.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.93%. Comparing base (69659c6) to head (afe3aa9).
⚠️ Report is 1 commits behind head on feat/backends.

Additional details and impacted files
@@               Coverage Diff               @@
##           feat/backends     #987    +/-   ##
===============================================
  Coverage          99.93%   99.93%            
===============================================
  Files                254      254            
  Lines              39820    40286   +466     
  Branches             837      835     -2     
===============================================
+ Hits               39795    40261   +466     
  Misses                25       25            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

All-backend test status (jax / torch)

Commit ade6e60524003e2000dde1287445a86353bb21dd

Green is achieved via the checked-in xfail-ledger (tests/backend_xfail.txt, applied xfail(strict=False)), so the metric to watch is the shrinking xfail count (burndown), not a raw pass count. A FAIL/ERR is an un-ledgered regression (reds the run). Because the ledger is non-strict, a now-passing ledgered test is a plain pass here (no per-push XPASS); burndown candidates -- in both directions -- are surfaced by the scheduled regen run, which rewrites the ledger from real outcomes. deferred is a separate burndown: tests skipped because they are unrunnable under the backend until the port is vectorized (see tests/backend_slow_skip.txt), e.g. the jax spherical-DF sampling/quadrature tests pending the Track F DF migration.

Overall: jax: 1056 passed · 266 xfail · 725 deferred | torch: 879 passed · 1164 xfail · 1 deferred

Ledger size: 2357 entries (jax=284, torch=2073).

Test shard jax torch
actionAngle ✅ 112 pass · 89 xfail ✅ 37 pass · 164 xfail
sphericaldf ✅ 164 pass · 26 xfail · 28 deferred ✅ 8 pass · 210 xfail
conversion + util + misc ✅ 85 pass · 6 xfail · 1 deferred ✅ 42 pass · 50 xfail
potential + scf + multipole — (no result) — (no result)
quantity + coords ✅ 287 pass · 49 xfail ✅ 206 pass · 130 xfail
orbit (energy/Jacobi + from_name) ✅ 0 pass · 0 xfail · 115 deferred ✅ 65 pass · 50 xfail
orbit + orbits (main) ✅ 0 pass · 0 xfail · 578 deferred ✅ 262 pass · 313 xfail
evolveddiskdf ✅ 35 pass · 0 xfail ✅ 32 pass · 3 xfail
jeans + dynamfric ✅ 17 pass · 2 xfail · 2 deferred ✅ 7 pass · 13 xfail · 1 deferred
qdf + pv2qdf + streamgapdf_impulse + noninertial ✅ 57 pass · 75 xfail · 1 deferred ✅ 14 pass · 119 xfail
streamgapdf ✅ 28 pass · 2 xfail ✅ 28 pass · 2 xfail
diskdf ✅ 129 pass · 0 xfail ✅ 112 pass · 17 xfail
streamdf + streamspraydf + streamTrack ✅ 142 pass · 17 xfail ✅ 66 pass · 93 xfail
Per-shard counts
Test shard backend pass xfail deferred XPASS fail error
actionAngle jax 112 89 0 0 0 0
actionAngle torch 37 164 0 0 0 0
sphericaldf jax 164 26 28 0 0 0
sphericaldf torch 8 210 0 0 0 0
conversion + util + misc jax 85 6 1 0 0 0
conversion + util + misc torch 42 50 0 0 0 0
potential + scf + multipole jax
potential + scf + multipole torch
quantity + coords jax 287 49 0 0 0 0
quantity + coords torch 206 130 0 0 0 0
orbit (energy/Jacobi + from_name) jax 0 0 115 0 0 0
orbit (energy/Jacobi + from_name) torch 65 50 0 0 0 0
orbit + orbits (main) jax 0 0 578 0 0 0
orbit + orbits (main) torch 262 313 0 0 0 0
evolveddiskdf jax 35 0 0 0 0 0
evolveddiskdf torch 32 3 0 0 0 0
jeans + dynamfric jax 17 2 2 0 0 0
jeans + dynamfric torch 7 13 1 0 0 0
qdf + pv2qdf + streamgapdf_impulse + noninertial jax 57 75 1 0 0 0
qdf + pv2qdf + streamgapdf_impulse + noninertial torch 14 119 0 0 0 0
streamgapdf jax 28 2 0 0 0 0
streamgapdf torch 28 2 0 0 0 0
diskdf jax 129 0 0 0 0 0
diskdf torch 112 17 0 0 0 0
streamdf + streamspraydf + streamTrack jax 142 17 0 0 0 0
streamdf + streamspraydf + streamTrack torch 66 93 0 0 0 0

…r _backend_dtype)

The all-backend coverage shard left three patch lines uncovered:
* _coerce.py promote_scalars: the asarray(device=) try/except fallback (the
  jax-string-'cpu' guard) is dead for current jax/torch (their .device is an
  object xp.asarray accepts), so promote it to delegate to asarray_on_device --
  the same helper coerce_coords uses -- which handles device + numpy->backend
  dtype in one place. Removes the dead except; the delegation sits on the already-
  covered path.
* _namespaces._backend_dtype: the not-a-generic branch is genuinely reachable --
  numpy>=2.0 StringDType resolves to the Python str scalar, so
  numpy.issubdtype(StringDType, numpy.generic) is False and it must be returned
  unchanged (no backend has a same-named dtype). Add a test exercising it under
  jax AND torch (self-skips on numpy<2.0).

numpy path byte-identical (both helpers early-return / pass-through on numpy).
Validated: 465 passed under the backend suite.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@jobovy jobovy merged commit e0d454d into feat/backends Jun 20, 2026
146 checks passed
@jobovy jobovy deleted the backend/central-scalar-coercion branch June 20, 2026 20:58
jobovy added a commit that referenced this pull request Jun 21, 2026
…backend

promote_scalars passed its inputs through unchanged when none was a backend
array to anchor on (`ref is None`), assuming "the namespace's functions handle
scalars". That holds for jax but NOT for torch: torch.cos/sqrt/... reject
numpy.float64 / python floats, so under a forced torch backend every
promote_scalars caller (the coords transforms cyl_to_rect / rect_to_cyl / ...,
OblateStaeckelWrapperPotential) crashed on all-numpy inputs. Coerce the operands
via coerce_coords in that branch instead.

Routing that branch through coerce_coords surfaced that #987's promote_scalars
refactor had silently dropped the device-reject fallback (the device-less
asarray retry when a namespace rejects the ref's .device value -- array-api jax
exposes .device as the string 'cpu' and jnp.asarray(device='cpu') raises
ValueError), leaving its test a no-op (the mock ref is no longer detected as a
backend array after #987's is_backend_array switch). Restore the fallback in
asarray_on_device (catch TypeError / ValueError -> device-less asarray; a
genuine dtype error re-raises from the fallback so it is not masked) and rewrite
the test to exercise asarray_on_device directly and deterministically.

The numpy path is byte-identical (the `xp is numpy` guard short-circuits, and
asarray_on_device's device branch is only taken when a backend array supplies a
device). jax value-identical under x64. Fixes the migrated RotateAndTilt /
Offset / OblateStaeckel / Kuzmin wrapper torch entries that route coordinates
through these transforms, plus 11 test_coords and 11 test_quantity torch cases.
New tests/test_backend_coerce.py covers the coercion branch.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant