Cautious optimizers

[fps]

Motivation and description

In https://arxiv.org/abs/2411.16085 the authors proposed "Cautious Optimizers: Improving Training with One Line of Code". Would you be interested in having this in Optimisers.jl? If so, since this modificaiton can be applied to many optimization algorithms I wonder how to best integrate it?

Kind regards

Possible Implementation

No response