OptimizationFunction uses reverse diff despite being told to use finite diff

**Describe the bug 🐞**
When passing in `AutoFiniteDiff` to a nonlinear constrained optimization problem, it seems like `Optimization` is still trying to use `ReverseDiff`. My guess is it's correctly using finite diff for the first order diff in the objective but trying to use reverse diff for the second order diff in the constraints.

**Expected behavior**

The optimization should not try to use any autodiff backend.

**Minimal Reproducible Example 👇**

```julia
using Optimization, OptimizationOptimJL

# Try to make a "it just works" nonlinear constrained optimization function like fmincon in MATLAB
function fmincon(objective, constraint, x0, p = nothing; kwargs...)
    f = OptimizationFunction(
        objective,
        Optimization.AutoFiniteDiff();
        cons = constraint,
    )
    prob = OptimizationProblem(f, x0, p; kwargs...)
    return solve(prob, IPNewton())
end

function break_autodiff!(x)
    x1, x2 = x
    # Break every form of autodiff
    x[1]::Float64 = x1
    return nothing
end

function objective(x, p)
    break_autodiff!(x)
    return (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
end

function constraint(res, x, p)
    break_autodiff!(x)
    res[1] = x[1]^2 + x[2]^2
    res[2] = x[1] * x[2]
    return nothing
end

p = ones(2)
x0 = zeros(2)
lcons = [-Inf, -1.0]
ucons = [0.8, 2.0]

sol = fmincon(objective, constraint, x0, p; lcons, ucons)
```

**Error & Stacktrace ⚠️**

```julia
ERROR: LoadError: MissingBackendError: Failed to use AutoReverseDiff().
Backend package is probably not loaded. To fix this, try to run

    import ReverseDiff

Stacktrace:
  [1] _prepare_pullback_aux(::DifferentiationInterface.PullbackFast, f::Function, backend::AutoReverseDiff{false}, x::Vector{Float64}, ty::Tuple{Bool}, contexts::DifferentiationInterface.Constant{Vector{Float64}})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/RHyMm/src/first_order/pullback.jl:151
  [2] prepare_pullback(f::typeof(objective), backend::AutoReverseDiff{false}, x::Vector{Float64}, ty::Tuple{Bool}, contexts::DifferentiationInterface.Constant{Vector{Float64}})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/RHyMm/src/first_order/pullback.jl:108
  [3] prepare_gradient(f::typeof(objective), backend::AutoReverseDiff{false}, x::Vector{Float64}, contexts::DifferentiationInterface.Constant{Vector{Float64}})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/RHyMm/src/first_order/gradient.jl:70
  [4] _prepare_hessian_aux(::Val{1}, f::typeof(objective), backend::DifferentiationInterface.SecondOrder{AutoFiniteDiff{…}, AutoReverseDiff{…}}, x::Vector{Float64}, contexts::DifferentiationInterface.Constant{Vector{…}})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/RHyMm/src/second_order/hessian.jl:90
  [5] prepare_hessian(f::typeof(objective), backend::DifferentiationInterface.SecondOrder{AutoFiniteDiff{…}, AutoReverseDiff{…}}, x::Vector{Float64}, contexts::DifferentiationInterface.Constant{Vector{…}})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/RHyMm/src/second_order/hessian.jl:76
  [6] instantiate_function(f::OptimizationFunction{…}, x::Vector{…}, adtype::AutoFiniteDiff{…}, p::Vector{…}, num_cons::Int64; g::Bool, h::Bool, hv::Bool, fg::Bool, fgh::Bool, cons_j::Bool, cons_vjp::Bool, cons_jvp::Bool, cons_h::Bool, lag_h::Bool)
    @ OptimizationBase ~/.julia/packages/OptimizationBase/aAhPJ/src/OptimizationDIExt.jl:64
  [7] instantiate_function
    @ ~/.julia/packages/OptimizationBase/aAhPJ/src/OptimizationDIExt.jl:17 [inlined]
  [8] #instantiate_function#40
    @ ~/.julia/packages/OptimizationBase/aAhPJ/src/OptimizationDIExt.jl:274 [inlined]
  [9] instantiate_function
    @ ~/.julia/packages/OptimizationBase/aAhPJ/src/OptimizationDIExt.jl:267 [inlined]
 [10] OptimizationCache(prob::OptimizationProblem{…}, opt::IPNewton{…}; callback::Function, maxiters::Nothing, maxtime::Nothing, abstol::Nothing, reltol::Nothing, progress::Bool, structural_analysis::Bool, manifold::Nothing, kwargs::@Kwargs{})
    @ OptimizationBase ~/.julia/packages/OptimizationBase/aAhPJ/src/cache.jl:45
 [11] OptimizationCache
    @ ~/.julia/packages/OptimizationBase/aAhPJ/src/cache.jl:25 [inlined]
 [12] #__init#2
    @ ~/.julia/packages/OptimizationOptimJL/BIkTp/src/OptimizationOptimJL.jl:109 [inlined]
 [13] __init
    @ ~/.julia/packages/OptimizationOptimJL/BIkTp/src/OptimizationOptimJL.jl:80 [inlined]
 [14] #init#661
    @ ~/.julia/packages/SciMLBase/M57fR/src/solve.jl:172 [inlined]
 [15] init
    @ ~/.julia/packages/SciMLBase/M57fR/src/solve.jl:170 [inlined]
 [16] solve(::OptimizationProblem{…}, ::IPNewton{…}; kwargs::@Kwargs{})
    @ SciMLBase ~/.julia/packages/SciMLBase/M57fR/src/solve.jl:94
 [17] solve(::OptimizationProblem{true, OptimizationFunction{…}, Vector{…}, Vector{…}, Nothing, Nothing, Nothing, Vector{…}, Vector{…}, Nothing, @Kwargs{}}, ::IPNewton{typeof(Optim.backtrack_constrained_grad), Symbol})
    @ SciMLBase ~/.julia/packages/SciMLBase/M57fR/src/solve.jl:91
 [18] fmincon(objective::Function, constraint::Function, x0::Vector{Float64}, p::Vector{Float64}; kwargs::@Kwargs{lcons::Vector{Float64}, ucons::Vector{Float64}})
    @ Main /Systems/blah.jl:11
 [19] top-level scope
    @ /Systems/blah.jl:37
 [20] include(fname::String)
    @ Base.MainInclude ./client.jl:489
 [21] top-level scope
    @ REPL[3]:1
in expression starting at /Systems/blah.jl:37
Some type information was truncated. Use `show(err)` to see complete types.
```

**Environment (please complete the following information):**

  - Output of `using Pkg; Pkg.status()`

```julia
Status `/tmp/jl_QXU18c/Project.toml`
  [7f7a1694] Optimization v4.0.2
  [36348300] OptimizationOptimJL v0.4.0
```

  - Output of `using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)`

```julia
Status `/tmp/jl_QXU18c/Manifest.toml`
  [47edcb42] ADTypes v1.9.0
  [1520ce14] AbstractTrees v0.4.5
  [7d9f7c33] Accessors v0.1.38
  [79e6a3ab] Adapt v4.0.4
  [dce04be8] ArgCheck v2.3.0
  [4fba245c] ArrayInterface v7.16.0
  [a9b6321e] Atomix v0.1.0
  [198e06fe] BangBang v0.4.3
  [9718e550] Baselet v0.1.1
  [fa961155] CEnum v0.5.0
  [d360d2e6] ChainRulesCore v1.25.0
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.1
  [34da2185] Compat v4.16.0
  [a33af91c] CompositionsBase v0.1.2
  [88cd18e8] ConsoleProgressMonitor v0.1.2
  [187b0558] ConstructionBase v1.5.8
  [6add18c4] ContextVariablesX v0.1.3
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [244e2a9f] DefineSingletons v0.1.2
  [8bb1440f] DelimitedFiles v1.9.1
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.6.5
  [ffbed154] DocStringExtensions v0.9.3
  [4e289a0a] EnumX v1.0.4
  [e2ba6199] ExprTools v0.1.10
⌅ [6b7a57c9] Expronicon v0.8.5
  [cc61a311] FLoops v0.2.2
  [b9860ae5] FLoopsBase v0.1.1
  [9aa1b823] FastClosures v0.3.2
  [1a297f60] FillArrays v1.13.0
  [6a86dc24] FiniteDiff v2.24.0
  [f6369f11] ForwardDiff v0.10.36
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.1.6
  [22cec73e] InitialValues v0.3.1
  [3587e190] InverseFunctions v0.1.17
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.6.0
  [b14d175d] JuliaVariables v0.2.4
  [63c18a36] KernelAbstractions v0.9.27
  [5be7bae1] LBFGSB v0.4.1
  [929cbde3] LLVM v9.1.2
  [1d6d02ad] LeftChildRightSiblingTrees v0.2.0
  [d3d80556] LineSearches v7.3.0
  [2ab3a3ac] LogExpFunctions v0.3.28
  [e6f89c97] LoggingExtras v1.0.3
  [d8e11817] MLStyle v0.4.17
  [f1d291b0] MLUtils v0.4.4
  [1914dd2f] MacroTools v0.5.13
  [128add7d] MicroCollections v0.2.0
  [e1d29d7a] Missings v1.2.0
  [d41bc354] NLSolversBase v7.8.3
  [872c559c] NNlib v0.9.24
  [77ba4419] NaNMath v1.0.2
  [71a1bf82] NameResolution v0.1.5
  [429524aa] Optim v1.9.4
  [7f7a1694] Optimization v4.0.2
  [bca83a33] OptimizationBase v2.2.1
  [36348300] OptimizationOptimJL v0.4.0
  [bac558e1] OrderedCollections v1.6.3
  [90014a1f] PDMats v0.11.31
  [65ce6f38] PackageExtensionCompat v1.0.2
  [d96e819e] Parameters v0.12.3
  [85a6dd25] PositiveFactorizations v0.2.4
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [8162dcfd] PrettyPrint v0.2.0
  [33c8b6b6] ProgressLogging v0.1.4
  [92933f4c] ProgressMeter v1.10.2
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.27.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [0bca4576] SciMLBase v2.55.0
  [c0aeaf25] SciMLOperators v0.3.10
  [53ae85a6] SciMLStructures v1.5.0
  [efcf1570] Setfield v1.1.1
  [605ecd9f] ShowCases v0.1.0
  [699a6c99] SimpleTraits v0.9.4
  [a2af1166] SortingAlgorithms v1.2.1
  [9f842d2f] SparseConnectivityTracer v0.6.6
  [0a514795] SparseMatrixColorings v0.4.5
  [276daf66] SpecialFunctions v2.4.0
  [171d559e] SplittablesBase v0.1.15
  [90137ffa] StaticArrays v1.9.7
  [1e83bf80] StaticArraysCore v1.4.3
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.3
  [2efcf032] SymbolicIndexingInterface v0.3.31
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [5d786b92] TerminalLoggers v0.1.7
  [28d57a85] Transducers v0.4.82
  [3a884ed6] UnPack v1.0.2
  [013be700] UnsafeAtomics v0.2.1
  [d80eeb9a] UnsafeAtomicsLLVM v0.2.1
  [dad2f222] LLVMExtra_jll v0.0.34+0
  [81d17ec3] L_BFGS_B_jll v3.0.1+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
```

  - Output of `versioninfo()`

```julia
Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 32 × AMD EPYC 7R13 Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 32 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_PKG_DEVDIR = /ubuntu/github
```

**Additional context**

Add any other context about the problem here.


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OptimizationFunction uses reverse diff despite being told to use finite diff #119

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

OptimizationFunction uses reverse diff despite being told to use finite diff #119

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions