Combining a `MatrixOperator` `mass_matrix` with a `jac_prototype`

[bmxam]

Describe the bug 🐞

Using OrdinaryDiffEq, I’m trying to solve an ODE involving a mass matrix that (may) evolve depending on the solution, the time, the parameters. So I set up a SciMLOperators.MatrixOperator and provided it to the ODEFunction. So far, so good.

The problem I’m facing is that if I provide a jac_prototype (of my rhs function) to the ODEFunction, then the solve errors (see stack below). Without jac_prototype, on a MWE, everything is fine.

It may be related to #2000 .

Expected behavior

The call to solve should not error.

Minimal Reproducible Example 👇

using OrdinaryDiffEq
using SparseArrays
using SparseDiffTools

const M_mwe = sparse([1.0 2.0; 0.0 1.0])

function update_func(A, u, p, t)
    return M_mwe
end

function run()
    mass_matrix = MatrixOperator(M_mwe; update_func = update_func)
    f!(du, u, p, t) = du .= u
    alg = ImplicitEuler()
    tspan = (0.0, 1.0)
    u0 = ones(2)

    # Without jac_prototype
    odeFunction = ODEFunction(f!; mass_matrix)
    prob = ODEProblem(odeFunction, u0, tspan)
    solve(prob, alg) #-------> OK

    # With jac_prototype
    jac_prototype = sparse([1.0 0.0; 0.0 1.0])
    colorvec = [1, 1]
    odeFunction = ODEFunction(f!; mass_matrix, jac_prototype, colorvec)
    prob = ODEProblem(odeFunction, u0, tspan)
    solve(prob, alg) #-------> KO
end

run()

Error & Stacktrace ⚠️

MethodError: no method matching keys(::MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}})
The function `keys` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  keys(::Cmd)
   @ Base process.jl:716
  keys(::Base.TermInfo)
   @ Base terminfo.jl:232
  keys(::LibGit2.GitTree)
   @ Revise .julia/packages/Revise/p9Zlq/src/git.jl:52
  ...

Stacktrace:
  [1] pairs(collection::MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}})
    @ Base ./abstractdict.jl:178
  [2] findall(testf::ComposedFunction{typeof(!), typeof(iszero)}, A::MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}})
    @ Base ./array.jl:2700
  [3] prepare_user_sparsity(ad_alg::AutoForwardDiff{nothing, ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}}, prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem})
    @ OrdinaryDiffEqDifferentiation .julia/packages/OrdinaryDiffEqDifferentiation/DE4z5/src/alg_utils.jl:120
  [4] prepare_alg(alg::ImplicitEuler{0, AutoForwardDiff{nothing, Nothing}, Nothing, NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}, Nothing}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!)}, u0::Vector{Float64}, p::SciMLBase.NullParameters, prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem})
    @ OrdinaryDiffEqDifferentiation .julia/packages/OrdinaryDiffEqDifferentiation/DE4z5/src/alg_utils.jl:53
  [5] solve_up(prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, sensealg::Nothing, u0::Vector{Float64}, p::SciMLBase.NullParameters, args::ImplicitEuler{0, AutoForwardDiff{nothing, Nothing}, Nothing, NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}, Nothing}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!)}; kwargs::@Kwargs{})
    @ DiffEqBase .julia/packages/DiffEqBase/qvEPa/src/solve.jl:1200
  [6] solve_up(prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, sensealg::Nothing, u0::Vector{Float64}, p::SciMLBase.NullParameters, args::ImplicitEuler{0, AutoForwardDiff{nothing, Nothing}, Nothing, NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}, Nothing}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!)})
    @ DiffEqBase .julia/packages/DiffEqBase/qvEPa/src/solve.jl:1183
  [7] solve(prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, args::ImplicitEuler{0, AutoForwardDiff{nothing, Nothing}, Nothing, NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}, Nothing}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!)}; sensealg::Nothing, u0::Nothing, p::Nothing, wrap::Val{true}, kwargs::@Kwargs{})
    @ DiffEqBase .julia/packages/DiffEqBase/qvEPa/src/solve.jl:1096
  [8] solve(prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.FullSpecialize, var"#f!#run##1", MatrixOperator{Float64, SparseMatrixCSC{Float64, Int64}, SciMLOperators.FilterKwargs{typeof(update_func), Tuple{}}, SciMLOperators.FilterKwargs{typeof(SciMLOperators.DEFAULT_UPDATE_FUNC), Tuple{}}}, Nothing, Nothing, Nothing, Nothing, Nothing, SparseMatrixCSC{Float64, Int64}, SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Vector{Int64}, Nothing, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, args::ImplicitEuler{0, AutoForwardDiff{nothing, Nothing}, Nothing, NLNewton{Rational{Int64}, Rational{Int64}, Rational{Int64}, Nothing}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!)})
    @ DiffEqBase .julia/packages/DiffEqBase/qvEPa/src/solve.jl:1086
  [9] run()
    @ Main ./REPL[11]:18
 [10] top-level scope
    @ REPL[12]:1

Environment (please complete the following information):

• Output of using Pkg; Pkg.status()

⌃ [1dea7af3] OrdinaryDiffEq v6.101.0
  [47a9eef4] SparseDiffTools v2.26.0
  [2f01184e] SparseArrays v1.12.0

• Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)

  [47edcb42] ADTypes v1.20.0
  [7d9f7c33] Accessors v0.1.43
  [79e6a3ab] Adapt v4.4.0
  [ec485272] ArnoldiMethod v0.4.0
  [4fba245c] ArrayInterface v7.22.0
  [4c555306] ArrayLayouts v1.12.2
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
⌃ [70df07ce] BracketingNonlinearSolve v1.3.0
  [2a0fbf3d] CPUSummary v0.2.7
  [d360d2e6] ChainRulesCore v1.26.0
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.1
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.18.1
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.6.0
  [adafc99b] CpuId v0.3.1
⌅ [864edb3b] DataStructures v0.18.22
⌃ [2b5f629d] DiffEqBase v6.176.0
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.7.12
  [ffbed154] DocStringExtensions v0.9.5
  [4e289a0a] EnumX v1.0.5
  [f151be2c] EnzymeCore v0.8.17
  [d4d017d3] ExponentialUtilities v1.28.0
  [e2ba6199] ExprTools v0.1.10
  [55351af7] ExproniconLite v0.10.14
  [7034ab61] FastBroadcast v0.3.5
  [9aa1b823] FastClosures v0.3.2
  [442a2c76] FastGaussQuadrature v1.1.0
  [a4df4552] FastPower v1.2.0
  [1a297f60] FillArrays v1.15.0
  [6a86dc24] FiniteDiff v2.29.0
  [f6369f11] ForwardDiff v1.3.0
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.2.0
  [c145ed77] GenericSchur v0.5.6
⌃ [86223c79] Graphs v1.13.1
  [615f187c] IfElse v0.1.1
  [d25df0c9] Inflate v0.1.5
  [3587e190] InverseFunctions v0.1.17
  [92d709cd] IrrationalConstants v0.2.6
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.7.1
  [ae98c720] Jieko v0.2.1
  [ba0b0d4f] Krylov v0.10.3
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.9.4
  [87fe0de2] LineSearch v0.1.4
  [d3d80556] LineSearches v7.5.1
⌃ [7ed4a6bd] LinearSolve v3.26.0
  [2ab3a3ac] LogExpFunctions v0.3.29
  [1914dd2f] MacroTools v0.5.16
  [d125e4d3] ManualMemory v0.1.8
  [bb5d69b7] MaybeInplace v0.1.4
  [2e0e35c7] Moshi v0.3.7
  [46d2c3a1] MuladdMacro v0.2.4
  [d41bc354] NLSolversBase v7.10.0
  [77ba4419] NaNMath v1.1.3
⌃ [8913a72c] NonlinearSolve v4.10.0
⌅ [be0214bd] NonlinearSolveBase v1.14.0
⌃ [5959db7a] NonlinearSolveFirstOrder v1.7.0
⌃ [9a2c21bd] NonlinearSolveQuasiNewton v1.8.0
⌃ [26075421] NonlinearSolveSpectralMethods v1.3.0
  [bac558e1] OrderedCollections v1.8.1
⌃ [1dea7af3] OrdinaryDiffEq v6.101.0
⌃ [89bda076] OrdinaryDiffEqAdamsBashforthMoulton v1.4.0
⌃ [6ad6398a] OrdinaryDiffEqBDF v1.9.0
⌃ [bbf590c4] OrdinaryDiffEqCore v1.26.2
⌃ [50262376] OrdinaryDiffEqDefault v1.7.0
⌃ [4302a76b] OrdinaryDiffEqDifferentiation v1.11.0
⌃ [9286f039] OrdinaryDiffEqExplicitRK v1.3.0
⌃ [e0540318] OrdinaryDiffEqExponentialRK v1.7.0
⌃ [becaefa8] OrdinaryDiffEqExtrapolation v1.7.0
⌃ [5960d6e9] OrdinaryDiffEqFIRK v1.14.0
⌃ [101fe9f7] OrdinaryDiffEqFeagin v1.3.0
⌃ [d3585ca7] OrdinaryDiffEqFunctionMap v1.4.0
⌃ [d28bc4f8] OrdinaryDiffEqHighOrderRK v1.4.0
⌃ [9f002381] OrdinaryDiffEqIMEXMultistep v1.6.0
⌃ [521117fe] OrdinaryDiffEqLinear v1.4.0
⌃ [1344f307] OrdinaryDiffEqLowOrderRK v1.5.0
⌃ [b0944070] OrdinaryDiffEqLowStorageRK v1.5.0
⌃ [127b3ac7] OrdinaryDiffEqNonlinearSolve v1.12.0
⌃ [c9986a66] OrdinaryDiffEqNordsieck v1.3.0
⌃ [5dd0a6cf] OrdinaryDiffEqPDIRK v1.5.0
⌃ [5b33eab2] OrdinaryDiffEqPRK v1.3.0
⌃ [04162be5] OrdinaryDiffEqQPRK v1.3.0
⌃ [af6ede74] OrdinaryDiffEqRKN v1.4.0
⌃ [43230ef6] OrdinaryDiffEqRosenbrock v1.14.0
⌃ [2d112036] OrdinaryDiffEqSDIRK v1.6.0
⌃ [669c94d9] OrdinaryDiffEqSSPRK v1.5.0
⌃ [e3e12d00] OrdinaryDiffEqStabilizedIRK v1.5.0
  [358294b1] OrdinaryDiffEqStabilizedRK v1.4.0
⌃ [fa646aed] OrdinaryDiffEqSymplecticRK v1.6.0
⌃ [b1df2697] OrdinaryDiffEqTsit5 v1.4.0
⌃ [79d7bb75] OrdinaryDiffEqVerner v1.5.0
  [65ce6f38] PackageExtensionCompat v1.0.2
  [d96e819e] Parameters v0.12.3
  [f517fe37] Polyester v0.7.18
  [1d0040c9] PolyesterWeave v0.2.2
  [d236fae5] PreallocationTools v0.4.34
  [aea7be01] PrecompileTools v1.3.3
  [21216c6a] Preferences v1.5.0
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.39.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.16
  [94e857df] SIMDTypes v0.1.0
⌃ [0bca4576] SciMLBase v2.99.0
  [19f34311] SciMLJacobianOperators v0.1.11
⌅ [c0aeaf25] SciMLOperators v0.4.0
  [431bcebd] SciMLPublic v1.0.0
  [53ae85a6] SciMLStructures v1.7.0
  [efcf1570] Setfield v1.1.2
⌃ [727e6d20] SimpleNonlinearSolve v2.7.0
  [699a6c99] SimpleTraits v0.9.5
  [ce78b400] SimpleUnPack v1.1.0
  [47a9eef4] SparseDiffTools v2.26.0
  [0a514795] SparseMatrixColorings v0.4.23
  [276daf66] SpecialFunctions v2.6.1
  [aedffcd0] Static v1.3.1
  [0d7ed370] StaticArrayInterface v1.8.0
  [90137ffa] StaticArrays v1.9.15
  [1e83bf80] StaticArraysCore v1.4.4
  [10745b16] Statistics v1.11.1
  [7792a7ef] StrideArraysCore v0.5.8
  [2efcf032] SymbolicIndexingInterface v0.3.46
  [8290d209] ThreadingUtilities v0.5.5
  [a759f4b9] TimerOutputs v0.5.29
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [19fa3120] VertexSafeGraphs v0.2.0
  [1d5cc7b8] IntelOpenMP_jll v2025.2.0+0
  [856f044c] MKL_jll v2025.2.0+0
  [efe28fd5] OpenSpecFun_jll v0.5.6+0
  [1317d2d5] oneTBB_jll v2022.0.0+1
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.7.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [a63ad114] Mmap v1.11.0
  [ca575930] NetworkOptions v1.3.0
  [44cfe95a] Pkg v1.12.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [1a1011a3] SharedArrays v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.12.0
  [f489334b] StyledStrings v1.11.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.3.0+1
  [deac9b47] LibCURL_jll v8.15.0+0
  [e37daf67] LibGit2_jll v1.9.0+0
  [29816b5a] LibSSH2_jll v1.11.3+1
  [14a3606d] MozillaCACerts_jll v2025.5.20
  [4536629a] OpenBLAS_jll v0.3.29+0
  [05823500] OpenLibm_jll v0.8.7+0
  [458c3c95] OpenSSL_jll v3.5.4+0
  [bea87d4a] SuiteSparse_jll v7.8.3+2
  [83775a58] Zlib_jll v1.3.1+2
  [8e850b90] libblastrampoline_jll v5.15.0+0
  [8e850ede] nghttp2_jll v1.64.0+1
  [3f19e933] p7zip_jll v17.7.0+0

• Output of versioninfo()

Julia Version 1.12.2
Commit ca9b6662be4 (2025-11-20 16:25 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × Intel(R) Xeon(R) w3-2423
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, sapphirerapids)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 12 virtual cores)

[ChrisRackauckas]

Generally it's better to use fully implicit DAE tools for this, so DFBDF or IDA.

[bmxam]

Thanks for the feedback, I will try to set up a DAE problem. My main motivation here was to replicate an existing in-house "ODE" algorithm with DiffEq. I have a few other "bugs" to report, always trying to solve an "ODE" with ODEProblem + nlsolver, do you want me to report them or I am just not targeting the right path in DiffEq?

Example of an other bug:

using OrdinaryDiffEq
using ForwardDiff
using NonlinearSolve

f!(du, u, p, y) = du .= u

function run()
    u0 = rand(4)
    prob = ODEProblem(f!, u0, (0.0, 1.0))
    solve(prob, ImplicitEuler(; nlsolve = NLNewton())) # OK

    solve(prob, ImplicitEuler(; nlsolve = NonlinearSolveAlg(SimpleNewtonRaphson()))) # KO
end

run()

[ChrisRackauckas]

NonlinearSolveAlg isn't quite ready for release yet. Hopefully in the next month it'll be the standard but there are some known issues with it around the edges that are high priority to handle first.

[bmxam]

Thanks for your answers.