Convergence criteria

[gaborantal]

Is your feature request related to a problem? Please describe.
Currently, only fixed number of iteration can be set to the optimizer. However, a convergence criteria would be useful.

Describe the solution you'd like
By using a convergence criteria, a maximum number of steps OR the convergence criteria could be used to stop the iteration (whichever happens first). The reason why the iteration stopped should be available after the execution.

Are you able and willing to implement this feature yourself and open a pull request?

• Yes, I can provide this feature.

If anyone can do this issue, that would be great, however, I'm also fine with doing it myself in a form of PR.

[till-m]

Hi @gaborantal,

you're not the first to bring this up and I can see the appeal.

In my personal opinion, since such things can be performed by using suggest-evaluate-register (#373) instead of optimizer.maximizer(), I would prefer to keep the API clean. I would be interested in what other people have to say though.

How would you determine that the optimization procedure has converged?

[bwheelz36]

I wouldn't mind adding optional convergence criteria as an argument that can be passed at instantiation.
At the moment, there is only really one - number of iterations. But we could also add:

• If x iterations have occured without at least y improvement (y should probably include both relative and absolute tolerance?)
• if a solution is found better than some value
• others?

[gaborantal]

* If `x` iterations have occured without at least `y` improvement (y should probably include both relative and absolute tolerance?)

This is exactly what we need. However, your other mentioned option also looks great, and might be useful.

[jacktang]

I wouldn't mind adding optional convergence criteria as an argument that can be passed at instantiation. At the moment, there is only really one - number of iterations. But we could also add:

• If x iterations have occured without at least y improvement (y should probably include both relative and absolute tolerance?)
• if a solution is found better than some value
• others?

From my project experiences, here are some termination criteria:

• input variables(X) tolerance based termination
• objective function value tolerance based termination
• max iteration / evaluation
• time based termination
• target value based termination

and we compose some terminations if possible