Skip to content

Commit 0d44597

Browse files
ChrisRackauckasChrisRackauckas
committed
Merge remote-tracking branch 'alonsoC1s/master'
2 parents 6864e55 + efa61e3 commit 0d44597

8 files changed

+3505
-4
lines changed

Diff for: ‎.buildkite/launch_benchmarks.yml.signature

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
1-
Salted__�c?�ު��DwL�J�/���)��|p��5 +c��\~o�.�� RR�hAZR��
2-
$'���� 6���!�m�^����S�j�Y�����
1+
Salted__�c?
2+
�ު��DwL�J�/���)��|p��5 +c��\~o�.�� RR�hAZR��
3+
$'���� 6���!�m�^����S�j�Y�����

Diff for: ‎.buildkite/run_benchmark.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ steps:
1919
- JuliaCI/julia#v1:
2020
version: "1.10"
2121
- staticfloat/sandbox:
22-
rootfs_url: "https://github.com/thazhemadam/openmodelica-rootfs-image/releases/download/v1.23.0/rootfs-openmodelica-v1.23.0.amd64.tar.gz"
23-
rootfs_treehash: "82970243dc4f188e599a976abc20951f4aba2912"
22+
rootfs_url: "https://github.com/thazhemadam/ScimlBenchmarks-rootfs-image/releases/download/rootfs-image/rootfs-script.amd64.tar.gz"
23+
rootfs_treehash: "943bde824554f0afad5a939341f05e3bdb393051"
2424
uid: 1000
2525
gid: 1000
2626
workspaces:

Diff for: ‎benchmarks/OptimizationCUTEst/CUTEst_bounded.jmd

+141
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
---
2+
title: CUTEst Bounded Constrained Nonlinear Optimization Benchmarks
3+
author: Alonso M. Cisneros
4+
---
5+
6+
# Introduction
7+
8+
CUTEst, the Constraind and Unconstrained Testing Environment is, as the name suggests is a
9+
collection of around 1500 problems for general nonlinear optimization used to test
10+
optimization routines. The wrapper
11+
[CUTEst.jl](https://github.com/JuliaSmoothOptimizers/CUTEst.jl) provides convenient access
12+
to the problem collection, which we can leverage to test the optimizers made available by
13+
Optimization.jl.
14+
15+
This benchmark uses the following packages:
16+
17+
```julia
18+
using Optimization
19+
using OptimizationNLPModels
20+
using CUTEst
21+
using OptimizationOptimJL
22+
using Ipopt
23+
using OptimizationMOI
24+
using OptimizationMOI: MOI as MOI
25+
# Analysis and plotting
26+
using DataFrames
27+
using Plots
28+
using StatsPlots
29+
```
30+
31+
# Benchmarks
32+
33+
We will be testing the [Ipopt]() and the [LBFGS]() optimizers on these classes of
34+
problems.
35+
36+
```julia
37+
optimizers = [
38+
Optimization.LBFGS(),
39+
MOI.OptimizerWithAttributes(Ipopt.Optimizer, "print_level" => 0)
40+
]
41+
42+
function get_stats(sol, ::Optimization.LBFGS)
43+
return (length(sol.u), sol.stats.time, "LBFGS", Symbol(sol.retcode))
44+
end
45+
46+
function get_stats(sol, ::OptimizationMOI.MOI.OptimizerWithAttributes)
47+
return (length(sol.u), MOI.get(sol.original.model, MOI.SolveTimeSec()),
48+
"Ipopt", Symbol(sol.retcode))
49+
end
50+
51+
function run_benchmarks(problems, optimizers)
52+
problem = String[]
53+
n_vars = Int64[]
54+
secs = Float64[]
55+
solver = String[]
56+
retcode = Symbol[]
57+
58+
optz = length(optimizers)
59+
n = length(problems)
60+
61+
@info "here 1"
62+
63+
broadcast(c -> sizehint!(c, optz * n), [problem, n_vars, secs, solver, retcode])
64+
65+
@info "here 2"
66+
67+
for prob_name in problems
68+
@info prob_name
69+
nlp_prob = CUTEstModel(prob_name)
70+
prob = OptimizationNLPModels.OptimizationProblem(nlp_prob, Optimization.AutoForwardDiff())
71+
for optimizer in optimizers
72+
sol = solve(prob, optimizer; maxiters = 1e7)
73+
74+
@info "Solved $(prob_name) with $(optimizer)"
75+
vars, time, alg, code = get_stats(sol, optimizer)
76+
77+
push!(problem, prob_name)
78+
push!(n_vars, vars)
79+
push!(secs, time)
80+
push!(solver, alg)
81+
push!(retcode, code)
82+
83+
end
84+
finalize(nlp_prob)
85+
end
86+
87+
return DataFrame(problem = problem, n_vars = n_vars, secs = secs, solver = solver,
88+
retcode = retcode)
89+
end
90+
```
91+
92+
## Equality/Inequality constrained problems with bounded variables
93+
94+
Now we analyze the subset of problems with equality/inequality constraints and whose
95+
variables are bounded. There are 666 such problems.
96+
97+
The following figure shows the results of the same benchmarks previously described for the
98+
problems on this section.
99+
100+
```julia
101+
@info "before"
102+
eq_bou_problems = CUTEst.select(min_con=1, only_equ_con=true, only_free_var=false)
103+
@info "after1"
104+
105+
# Analysis
106+
eq_bou_results = run_benchmarks(eq_bou_problems, optimizers)
107+
@info "after2"
108+
109+
@df eq_bou_results scatter(:n_vars, :secs,
110+
group = :solver,
111+
xlabel = "n. variables",
112+
ylabel = "secs.",
113+
title = "Time to solution by optimizer and number of vars",
114+
)
115+
@info "after3"
116+
```
117+
118+
Next, we examine the same relationship for problems with inequality-constrained problems,
119+
of which there are 244.
120+
121+
```julia
122+
@info "after4"
123+
neq_bou_problems = CUTEst.select(min_con=1, only_ineq_con=true, only_free_var=false)
124+
@info "after5"
125+
126+
# Analysis
127+
neq_bou_results = run_benchmarks(neq_bou_problems, optimizers)
128+
@info "after6"
129+
130+
@df neq_bou_results scatter(:n_vars, :secs,
131+
group = :solver,
132+
xlabel = "n. variables",
133+
ylabel = "secs.",
134+
title = "Time to solution by optimizer and number of vars",
135+
)
136+
```
137+
138+
```julia, echo = false
139+
using SciMLBenchmarks
140+
SciMLBenchmarks.bench_footer(WEAVE_ARGS[:folder],WEAVE_ARGS[:file])
141+
```

Diff for: ‎benchmarks/OptimizationCUTEst/CUTEst_quadratic.jmd

+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
---
2+
title: CUTEst Quadratic Programming with Linear Constraints Benchmarks
3+
author: Alonso M. Cisneros
4+
---
5+
6+
# Introduction
7+
8+
CUTEst, the Constraind and Unconstrained Testing Environment is, as the name suggests is a
9+
collection of around 1500 problems for general nonlinear optimization used to test
10+
optimization routines. The wrapper
11+
[CUTEst.jl](https://github.com/JuliaSmoothOptimizers/CUTEst.jl) provides convenient access
12+
to the problem collection, which we can leverage to test the optimizers made available by
13+
Optimization.jl.
14+
15+
This benchmark uses the following packages:
16+
17+
```julia
18+
using Optimization
19+
using OptimizationNLPModels
20+
using CUTEst
21+
using OptimizationOptimJL
22+
using Ipopt
23+
using OptimizationMOI
24+
using OptimizationMOI: MOI as MOI
25+
# Analysis and plotting
26+
using DataFrames
27+
using Plots
28+
using StatsPlots
29+
```
30+
31+
# Benchmarks
32+
33+
We will be testing the [Ipopt]() and the [LBFGS]() optimizers on these classes of
34+
problems.
35+
36+
```julia
37+
optimizers = [
38+
Optimization.LBFGS(),
39+
MOI.OptimizerWithAttributes(Ipopt.Optimizer, "print_level" => 0)
40+
]
41+
42+
function get_stats(sol, ::Optimization.LBFGS)
43+
return (length(sol.u), sol.stats.time, "LBFGS", Symbol(sol.retcode))
44+
end
45+
46+
function get_stats(sol, ::OptimizationMOI.MOI.OptimizerWithAttributes)
47+
return (length(sol.u), MOI.get(sol.original.model, MOI.SolveTimeSec()),
48+
"Ipopt", Symbol(sol.retcode))
49+
end
50+
51+
function run_benchmarks(problems, optimizers)
52+
problem = String[]
53+
n_vars = Int64[]
54+
secs = Float64[]
55+
solver = String[]
56+
retcode = Symbol[]
57+
58+
optz = length(optimizers)
59+
n = length(problems)
60+
61+
broadcast(c -> sizehint!(c, optz * n), [problem, n_vars, secs, solver, retcode])
62+
63+
for prob_name in problems
64+
nlp_prob = CUTEstModel(prob_name)
65+
prob = OptimizationNLPModels.OptimizationProblem(nlp_prob, Optimization.AutoForwardDiff())
66+
for optimizer in optimizers
67+
sol = solve(prob, optimizer; maxiters = 1e7)
68+
69+
@info "Solved $(prob_name) with $(optimizer)"
70+
vars, time, alg, code = get_stats(sol, optimizer)
71+
72+
push!(problem, prob_name)
73+
push!(n_vars, vars)
74+
push!(secs, time)
75+
push!(solver, alg)
76+
push!(retcode, code)
77+
78+
end
79+
finalize(nlp_prob)
80+
end
81+
82+
return DataFrame(problem = problem, n_vars = n_vars, secs = secs, solver = solver,
83+
retcode = retcode)
84+
end
85+
```
86+
87+
# Quadratic programs with linear constraints
88+
89+
Lastly, we examine the problems with a quadratic objective function and only linear
90+
constraints. There are 252 such problems in the suite.
91+
92+
```julia
93+
quad_problems = CUTEst.select(objtype="quadratic", contype="linear")
94+
95+
# Analysis
96+
quad_results = run_benchmarks(quad_problems, optimizers)
97+
98+
@df neq_bou_results scatter(:n_vars, :secs,
99+
group = :solver,
100+
xlabel = "n. variables",
101+
ylabel = "secs.",
102+
title = "Time to solution by optimizer and number of vars",
103+
)
104+
```
105+
106+
```julia, echo = false
107+
using SciMLBenchmarks
108+
SciMLBenchmarks.bench_footer(WEAVE_ARGS[:folder],WEAVE_ARGS[:file])
109+
```

0 commit comments

Comments
 (0)