Objective
Bring OptimalControl back onto the current ecosystem. The package has been pinned at CTBase = "=0.18.8" / CTModels = "=0.10.1" while every dependency below it was restructured. This issue tracks the upgrade, the addition of CTLie as a new dependency, and the test rework that goes with them.
Out of scope, deliberately: docs/. A separate documentation PR follows — expect ~24 stale pages after this lands.
Why now
Two independent things happened underneath us.
The whole ecosystem adopted the Handbook modules.md rule — the package top level exports nothing, each submodule owns its API. Names are still reachable through the package root, so most imports don't break, but they are non-compliant and would break silently on the next internal move.
Three families of symbols physically changed package, and those do break:
| Was |
Is now |
CTSolvers.{Options, Strategies, Orchestration} |
CTBase.* |
CTFlows.{Hamiltonian, HamiltonianVectorField, VectorField} |
CTBase.Data.* |
CTFlows.{Lift, Poisson, ∂ₜ, @Lie} |
CTLie.* |
CTDirect.{AbstractDiscretizer, discretize} |
CTSolvers.DOCP.* |
45 of the ~180 symbols in src/imports/ no longer resolve where we look for them.
What changes for users
Three groups, all going into BREAKING.md:
- Differential geometry —
Lie(X, f) → ad(X, f); X ⋅ f → ad(X, f), removed with no alias; HamiltonianLift → CTLie.LiftedHamiltonianFunction, and it is no longer <: AbstractHamiltonian.
- Flow calls — the variable is now a mandatory keyword on
NonFixed problems (f(t0, x0, p0, tf; variable=λ), no positional slot); augment= → variable_costate=; constrained flows take constraint=/multiplier= keyword pairs instead of three positional functions; constructor keywords gain the is_ prefix.
Flow now requires an integrator using — using OrdinaryDiffEq (or OrdinaryDiffEqTsit5) alongside using OptimalControl. This is a deliberate choice, see below.
There is also new API worth exposing that we don't re-export today: the whole CTBase.Data type vocabulary the user writes flows against (OpenLoop, ClosedLoop, DynClosedLoop, PathConstraint, Multiplier, PseudoHamiltonian, …), plus method = :cpu | :gpu on every Flow constructor — so :cpu/:gpu finally means the same thing on the direct and the indirect side.
Two decisions worth surfacing here
Name collisions were resolved upstream, not arbitrated. 49 names are exported by more than one submodule. Rather than picking winners in OptimalControl — which would have been type piracy — the two families that were genuinely one concept were unified in their owning packages: state/control/costate/objective/time_grid and status/successful now extend the CTModels generics. Measured before/after: 23 → 31 shared generics, 26 → 18 distinct. times was deliberately not unified (CTModels' returns a TimesModel component, CTSolvers' a time grid).
Flow needs an integrator using. Keeping SciMLBase, DiffEqBase and OrdinaryDiffEqTsit5 as hard dependencies costs 35 packages, 40 % of the dependency graph, to save the user a single using. Measured, not guessed. Moving them out makes the story uniform with what we already do: you load a solver to solve, Plots to plot, an integrator to integrate.
Upstream — all clear
Both original blockers were released before this work started:
A third surfaced mid-migration and is now also resolved:
- CTBase.jl#515 —
OpenLoop should be unconditionally non-autonomous → shipped in CTBase 0.28.9-beta. Companion fix for CTFlows' own call sites: CTFlows.jl#378.
Non-blocking and tracked separately: CTFlows.jl#368, CTFlows.jl#369, CTSolvers.jl#189.
Tests
4 files needed substantive rewrites, 13 needed path fixes, and the new work roughly doubles test/problems/:
- indirect and direct problems written both with
@def and with the functional API, behind one interface so a single test body runs twice — seeded from the 25 indirect problems with reference solutions in CTFlows.jl/test/suite/integration/;
- 1-D = scalar contract tests per
Handbook/philosophy/dimension-and-shape.md, through both paths;
- the flow API surface of the new call convention, control-law kinds, constrained flows and pseudo-Hamiltonians — previously zero coverage;
- CPU/GPU routing assertions (CPU-runnable) plus guarded device runs, gated behind the
kkt self-hosted runner.
Full local suite is green: 2167 passed / 2 broken (expected downstream skips) / 0 failed, ~26 min.
Progress
All done. See #826 for the full PR.
Full analysis — migration map, API specification, decisions Q1–Q8 with their rationale, and the step-by-step work order — follows in a comment.
Objective
Bring OptimalControl back onto the current ecosystem. The package has been pinned at
CTBase = "=0.18.8"/CTModels = "=0.10.1"while every dependency below it was restructured. This issue tracks the upgrade, the addition of CTLie as a new dependency, and the test rework that goes with them.Out of scope, deliberately:
docs/. A separate documentation PR follows — expect ~24 stale pages after this lands.Why now
Two independent things happened underneath us.
The whole ecosystem adopted the Handbook
modules.mdrule — the package top level exports nothing, each submodule owns its API. Names are still reachable through the package root, so most imports don't break, but they are non-compliant and would break silently on the next internal move.Three families of symbols physically changed package, and those do break:
CTSolvers.{Options, Strategies, Orchestration}CTBase.*CTFlows.{Hamiltonian, HamiltonianVectorField, VectorField}CTBase.Data.*CTFlows.{Lift, Poisson, ∂ₜ, @Lie}CTLie.*CTDirect.{AbstractDiscretizer, discretize}CTSolvers.DOCP.*45 of the ~180 symbols in
src/imports/no longer resolve where we look for them.What changes for users
Three groups, all going into
BREAKING.md:Lie(X, f)→ad(X, f);X ⋅ f→ad(X, f), removed with no alias;HamiltonianLift→CTLie.LiftedHamiltonianFunction, and it is no longer<: AbstractHamiltonian.NonFixedproblems (f(t0, x0, p0, tf; variable=λ), no positional slot);augment=→variable_costate=; constrained flows takeconstraint=/multiplier=keyword pairs instead of three positional functions; constructor keywords gain theis_prefix.Flownow requires an integratorusing—using OrdinaryDiffEq(orOrdinaryDiffEqTsit5) alongsideusing OptimalControl. This is a deliberate choice, see below.There is also new API worth exposing that we don't re-export today: the whole
CTBase.Datatype vocabulary the user writes flows against (OpenLoop,ClosedLoop,DynClosedLoop,PathConstraint,Multiplier,PseudoHamiltonian, …), plusmethod = :cpu | :gpuon everyFlowconstructor — so:cpu/:gpufinally means the same thing on the direct and the indirect side.Two decisions worth surfacing here
Name collisions were resolved upstream, not arbitrated. 49 names are exported by more than one submodule. Rather than picking winners in OptimalControl — which would have been type piracy — the two families that were genuinely one concept were unified in their owning packages:
state/control/costate/objective/time_gridandstatus/successfulnow extend the CTModels generics. Measured before/after: 23 → 31 shared generics, 26 → 18 distinct.timeswas deliberately not unified (CTModels' returns aTimesModelcomponent, CTSolvers' a time grid).Flowneeds an integratorusing. KeepingSciMLBase,DiffEqBaseandOrdinaryDiffEqTsit5as hard dependencies costs 35 packages, 40 % of the dependency graph, to save the user a singleusing. Measured, not guessed. Moving them out makes the story uniform with what we already do: you load a solver tosolve,Plotsto plot, an integrator to integrate.Upstream — all clear
Both original blockers were released before this work started:
0.4.33-beta0.16.1-betaA third surfaced mid-migration and is now also resolved:
OpenLoopshould be unconditionally non-autonomous → shipped in CTBase0.28.9-beta. Companion fix for CTFlows' own call sites: CTFlows.jl#378.Non-blocking and tracked separately: CTFlows.jl#368, CTFlows.jl#369, CTSolvers.jl#189.
Tests
4 files needed substantive rewrites, 13 needed path fixes, and the new work roughly doubles
test/problems/:@defand with the functional API, behind one interface so a single test body runs twice — seeded from the 25 indirect problems with reference solutions inCTFlows.jl/test/suite/integration/;Handbook/philosophy/dimension-and-shape.md, through both paths;kktself-hosted runner.Full local suite is green: 2167 passed / 2 broken (expected downstream skips) / 0 failed, ~26 min.
Progress
Project.toml: compat, CTLie, extension triggers;.extraspins dropped.Pkg.resolve()clean, direct deps 21 → 17src/imports/(+ctlie.jl,adnlpmodels.jl) and add the extension-arming importssrc/helpers/+src/solve/(~15 sites)BREAKING.md+CHANGELOG.mdAll done. See #826 for the full PR.
Full analysis — migration map, API specification, decisions Q1–Q8 with their rationale, and the step-by-step work order — follows in a comment.