How to limit attemps?

[Akronae]

At first I really liked the idea of using cryptography instead of storing OTP attemps in DB, but how to protect from buteforce attacks?
Usually I'd expect any OTP attempt to be invalidated after let's say 5 tries.

But if we're not using any DB we can't do that, and let's say you run 20 API instances, and that the attacker is using a VPN that allows him to use 10k IPs. Consdering your only lever is to limit the request rate on any endpoint to let's say 1/s, the attacker can make up to 200k tries per second. Isn't that a lot?