Skip to content

Remove Tracker @test_broken guards (closes #1331)#1452

Merged
ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:cc/remove-tracker-broken-guards-1331
May 30, 2026
Merged

Remove Tracker @test_broken guards (closes #1331)#1452
ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:cc/remove-tracker-broken-guards-1331

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Please ignore until reviewed by @ChrisRackauckas.

Depends on SciML/OrdinaryDiffEq.jl#3663 (DiffEqBase v7.5.1). This PR will fail CI until that PR is merged and a new DiffEqBase patch is tagged.

Closes #1331.

Summary

  • Drops the five if backend_name == "Tracker" && VERSION >= v"1.12"; @test_broken false; continue; end guards in test/concrete_solve_derivatives.jl (save_idxs, save_everystep=false, non-integer saveat=2.3, VecOfArray, BouncingBall).
  • Bumps DiffEqBase compat to 7.5.1.

The issue was filed as "Tracker fails on Julia 1.12+", but the actual trigger is the RecursiveArrayTools v4 supertype change (AbstractVectorOfArray <: AbstractArray). On RAT v3, the Tracker @grad for solve_up fell through to convert(AbstractArray, sol) which stack'd sol.u into a flat matrix. On RAT v4, the new sol isa AbstractArray branch ships the nested Vector{Vector{Float64}} to Tracker, and sum(solve(...)) reduces element-wise into a vector, tripping Tracker.gradient's "Function output is not scalar" check. Full diagnosis on #1331.

The fix lives in DiffEqBase (SciML/OrdinaryDiffEq.jl#3663); this PR is the test-side cleanup confirming the previously-broken tests now pass.

Verification

With the DiffEqBase patch applied locally on Julia 1.12.6 / Tracker 0.2.38 / RAT 4.3.0:

Test Summary: | Pass  Total     Time
save_idxs     |    1      1  1m14.8s
Test Summary:        | Pass  Total   Time
save_everystep=false |    1      1  11.6s
Test Summary: | Pass  Total   Time
saveat=2.3    |    1      1  23.9s
Test Summary: | Pass  Total   Time
VecOfArray    |    1      1  14.7s
Test Summary: | Pass  Total   Time
BouncingBall  |    1      1  39.4s

Test plan

The guards in test/concrete_solve_derivatives.jl (lines 518, 548, 579,
611, 735) attributed five Tracker outer-AD failures to "Julia 1.12+",
but the actual trigger was DiffEqBase's Tracker `@grad` for `solve_up`
returning the nested `Vector{Vector{Float64}}` for an `ODESolution`
after RecursiveArrayTools v4 made `AbstractVectorOfArray <: AbstractArray`.
Tracker tracked the vector-of-vectors directly, and `sum(solve(...))`
reduced element-wise into a vector instead of a scalar, hitting
"Function output is not scalar" in `Tracker.gradient`'s losscheck.

DiffEqBase v7.5.1 (SciML/OrdinaryDiffEq.jl#3663) stacks
`AbstractVectorOfArray` returns into a flat matrix before tracking,
restoring the pre-RAT-v4 behavior. With that fix the five originally
failing tests (`save_idxs`, `save_everystep=false`, non-integer
`saveat=2.3`, `VecOfArray`, `BouncingBall`) pass on Julia 1.12.6 with
Tracker 0.2.38. Bump DiffEqBase compat to 7.5.1 and drop the guards.

Closes SciML#1331.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Local verification on Julia 1.12.6 + this PR branch (cc/remove-tracker-broken-guards-1331) shows the released DiffEqBase 7.5.1 does not actually fix #1331 — so this PR will still fail CI even on the new 7.5.1 release.

What I found

PR SciML/OrdinaryDiffEq.jl#3665 (merged 2026-05-21T13:48Z, released as DiffEqBase 7.5.1 at 14:08Z) was supposed to add a new if sol isa RecursiveArrayTools.AbstractVectorOfArray; return sol, pb_f; end branch before the existing if sol isa AbstractArray check in DiffEqBaseTrackerExt.jl. The original commit on that PR (61b0c61) included that branch. A follow-up commit on the same PR (139911b, "Update DiffEqBaseTrackerExt.jl") deleted it before the merge, leaving only an effectively-unreachable fallback change:

- return convert(AbstractArray, sol), pb_f
+ return sol, pb_f

Under RAT v4, AbstractVectorOfArray <: AbstractArray, so the existing if sol isa AbstractArray branch matches ODESolution first and still returns sol.u :: Vector{Vector{Float64}} to Tracker — the original bug. The fallback is never hit for ODESolution.

Reproduction with the released 7.5.1

Smoke MWE (Tracker.gradient(p -> sum(solve(remake(prob; p), Tsit5(); kw...)), p0)):

FAIL  NamedTuple()                => Function output is not scalar
FAIL  (save_idxs = [1],)          => Function output is not scalar
FAIL  (save_everystep = false,)   => Function output is not scalar
FAIL  (saveat = 2.3,)             => Function output is not scalar

This PR's first testset (save_idxs Testssave_idxs - Tracker) on Julia 1.12.6 / DiffEqBase 7.5.1 (released):

save_idxs - Tracker: Error During Test
  Got exception outside of a @test
  Function output is not scalar
  [...]
    [2] losscheck(x::Tracker.TrackedVector{Float64, Vector{Float64}})
      @ Tracker ~/.julia/packages/Tracker/dLAzA/src/back.jl:153

Proposed path forward

I've opened SciML/OrdinaryDiffEq.jl#3666, which restores the dropped if sol isa AbstractVectorOfArray; return sol, pb_f; end branch and bumps DiffEqBase to 7.5.2. Verified locally on Julia 1.12.6 with that patch dev'd in:

  • Smoke MWE: 4/4 PASS, all produce scalar tracked gradients
  • The 5 previously-@test_broken testsets in this PR (via a focused runner that replicates the prelude + the 5 testsets verbatim):
    Test Summary:              | Pass  Total     Time
    save_idxs Tests            |    2      2  1m24.3s
    save_everystep=false Tests |    2      2  16.4s
    Non-integer saveat Tests   |    2      2  28.2s
    VecOfArray Derivatives     |    2      2  16.2s
    BouncingBall ODE           |    3      3  45.7s
    

So:

  1. Land Restore AbstractVectorOfArray branch in Tracker @grad (RAT v4) OrdinaryDiffEq.jl#3666 and tag DiffEqBase 7.5.2.
  2. Bump this PR's DiffEqBase compat from 7.5.17.5.2.
  3. Re-run CI here; the five testsets should pass.

Recommend holding merge of this PR until 7.5.2 is released. /cc @ChrisRackauckas

DiffEqBase 7.5.5 restores the AbstractVectorOfArray branch in the Tracker
@Grad for solve_up, which is what makes the five previously-@test_broken
Tracker testsets pass under RecursiveArrayTools v4.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review May 30, 2026 20:04
@ChrisRackauckas ChrisRackauckas merged commit 2d48e6d into SciML:master May 30, 2026
18 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tracker gradient tests fail on Julia 1.12+

2 participants