Question: Wondering if DZOptimization.jl allows box constraints?

[donboyd5]

Hi. I really like the idea of DZOptimization.jl -- fast, low memory, lightweight, full user control over termination.

It would be super-valuable to me if it allows box constraints, especially on LBFGS. I see that the function signatures allow constraint_function!::C that returns a boolean, but I didn't see any examples of how to use it. Would you mind letting me know if it does support box constraints and if so, would you mind sharing an example of how to code them if you have one available?

Many thanks.

[dzhang314]

Hey @donboyd5, thank you for your interest in DZOptimization.jl! I'm glad you find the premise of the project appealing, and I apologize for the lack of documentation -- as with most research code, I got DZOptimization.jl working well enough for my own purposes, and haven't really maintained it since.

To answer your question, yes, the constraint_function! mechanism is designed to support box constraints and manifold constraints among many others. The basic idea is that a constraint function should accept a point::Array{T,N} and attempt to project that point onto the feasible region. It should return true if that projection succeeded; if false is returned, then DZOptimization.jl will retry with a smaller step from the last known feasible point until projection succeeds.

Implementing a box constraint in this framework is very simple: you simply clamp each value in the current point to lie within the box bounds. This never fails to project the current point into the feasible region, so you always return true. An example might look like this:

function box_constraint!(point::Array{Float64,N}) where {N}
    @simd ivdep for i = 1 : length(point)
        @inbounds point[i] = clamp(point[i], 0.0, 1.0)
    end
    return true
end

Of course, you'll have to modify this example for bounds other than [0, 1].

As you can see, this mechanism is very flexible and allows DZOptimization.jl to handle any feasible region geometry that you can write a projection function for. For a more advanced example, you could imagine running an optimization algorithm on a manifold defined by an equation f(x, y, z, ...) == 0; in that context, your constraint_function! could run a root-finding algorithm, like Newton's method, on the equation f(x, y, z, ...) == 0 from the given initial point, and return false if root-finding fails to converge. I hope you'll agree that this is consistent with the design goals of DZOptimization.jl to make no assumptions about the nature of the problem you want to solve, and to provide the user with as much fine-grained control as possible.

Hope this helps!

[donboyd5]

Wow, David, this is really educational and also a bit magical. It works perfectly. I am going to try this out on some of my large-scale problems.

Thanks!