Add generic AutoDespecialize solver path - #4216
Conversation
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
The first eight completed CI failures are package-server registry lag, before any feature test runs. Representative jobs:
Each fresh runner's package-server registry reports SciMLBase versions only through 3.45.0 and then exits during resolution because this PR correctly requires 3.46.0: SciMLBase 3.46.0 is merged and registered through SciML/SciMLBase.jl#1514 and JuliaRegistries/General#164118. The exact registered source was used for the local failing-before/passing-after, QA, InterfaceII, and benchmark validations. I am leaving the 3.46 compatibility floor intact and will rerun the failed jobs after the package-server snapshot contains the registered release. |
|
Package-server propagation update (2026-08-11): A fresh local probe using the package server eager registry preference now resolves and installs The same probe with the default registry preference, after The default registry tarball contains versions only through 3.44, whereas the eager tarball contains 3.46. I am therefore not retrying the failed standard CI jobs yet; they would still resolve against the stale default snapshot. Registration: JuliaRegistries/General#164118 |
What changed
This adds DiffEqBase's generic
AutoDespecializesolver path on top of SciMLBase 3.46.pis stored inSciMLBase.DespecializedParameters, solver-facing function calls use a stable barrier type, and the original concrete parameter is recovered only when the user function is invoked. The underlying SciMLBase implementation provides that call barrier for every built-in SciMLFunction family. DiffEqBase promotes the differential-equation families that reach its solve layer: ODE, DAE, DDE, SDE, SDDE, and split problems; ODE auxiliary functions and stochastic diffusion functions cross the same barrier.AutoSpecializeis unchanged.AutoRespecializeremains the constrained, non-dynamic policy (withAutoDePSpecializeas its deprecated alias). OrdinaryDiffEq exposes all three policies as documented local aliases of the SciMLBase types. Common explicit and implicit solver workloads now precompile theAutoDespecializepath.This is stacked on the independent docs prerequisite #4210 and requires the merged SciMLBase implementation and registration from SciML/SciMLBase.jl#1514 and JuliaRegistries/General#164118.
Please ignore this draft until it has been reviewed by @ChrisRackauckas.
Verification
Discriminating test
I ran the exact added
lib/DiffEqBase/test/despecialized_p_test.jlagainst unmodified DiffEqBase 7.13.1 with registered SciMLBase 3.46.0:The failures showed that the two parameter layouts retained different concrete parameter/problem types. On this branch, the exact same test passes:
The test checks same concrete problem/function types across parameter layouts, original-parameter recovery, RHS/Jacobian/time-gradient dispatch, and DAE/DDE/SDE/SDDE drift and diffusion dispatch.
Local test/build results
Runic, typos over the additions/new tests, and
git diff --checkpass locally.Compile-time/runtime benchmark
Fresh Julia processes, one thread pinned to one CPU, Tsit5, 1,000 fixed steps. The second problem changes the concrete parameter layout after compiling/solving the first.
For this generic-function workload,
AutoDespecializereduced second-layout compilation by 94.3% versusAutoSpecialize, at a 0.259 ms absolute cost per 1,000-step warm solve (1.88x).For generated MTK systems the function recompilation dominates, so the result is more modest. At 64 states,
AutoDespecializecompiled the second layout in 0.572 s versus 0.550 s forAutoSpecializeand 1.457 s forFullSpecialize; warm medians were 1.203, 0.827, and 0.890 ms. At 512 states, the compile times were 0.575, 0.580, and 1.400 s; warm medians were 3.880, 5.273, and 3.331 ms. This shows the intended solver-code reuse versusFullSpecialize, while the comparison with existingAutoSpecializeis workload-dependent and the warm dynamic-dispatch cost is contained but measurable.Not verified locally
GPU paths and external downstream packages were not run. This adds no dependency and does not change a dependency license boundary.