Description
What's missing?
The sqlx
driver for rocket_db_pools
currently crashes at startup if your database is offline or if anything else fails -- sslrootcert is invalid/missing, user doesn't exist, password wrong, etc etc. This is sometimes useful in development as you are reminded to start your database if you forgot, but a log would work just as well. In production it is a bad choice.
The deadpool
crate's second-top-billed feature is that pool creation never fails even if the database is unavailable:
Identical startup and runtime behaviour. When writing long running application there usually should be no difference between startup and runtime if a database connection is temporarily not available. Nobody would expect an application to crash if the database becomes unavailable at runtime. So it should not crash on startup either. Creating the pool never fails and errors are only ever returned when calling Pool::get().
I'd like that, basically for the reasoning given there. It is needlessly crashy to do so only at startup.
To motivate somewhat more, if you are not convinced:
- I am currently debugging the provision + rotation of database credentials and TLS certificates to a Rocket application, and it is quite annoying having to restart a crash-loop-backoff rocket application deployment to see if one of the changes I have made works. K8s is able to update a rotated sslrootcert certificate file in-place without rebooting a container -- because of the crash behaviour, Rocket does not.
- It would be much easier to recover from a failed database cluster if a crashed or recovering database didn't completely bring down everything else that used it. The startup behaviour means all the alarms go off during a deployment when in fact your database went down an hour ago and you already know that. It means you have to reboot more things more times, while you're already in an emergency, but also, you must wait until your DB has finished recovering from a backup before rebooting everything else, so this either
- forces you to be really careful not to deploy to your HTTP servers, or allow them to crash, during a DB meltdown
- or necessarily slows down the app recovery, by forcing it to happen serially
Ideal Solution
The sqlx implementation of rocket_db_pools' Database trait uses Pool::connect_with
instead of Pool::connect_lazy_with. You can make a one-line change to switch to connect_lazy_with
, and make it behave like deadpool.
You could also make it configurable, but if people really want their app to crash in this case, they can follow deadpool's advice and add a fairing that tries to make a connection and crashes the app. People writing .unwrap()
in a fairing is probably easier than making this configurable.
Why can't this be implemented outside of Rocket?
It has been implemented outside of rocket. The deadpool backend behaves this way, I presume. But rocket_db_pools does not configure sqlx to do it.
Are there workarounds usable today?
To work around this limitation, you could probably use a custom Database or Pool implementation, but you would need to make your own wrapper type for sqlx::Pool
in order to get around the fact that rocket_db_pools implements its traits on that. I will probably do this myself for now, if it turns out to be easy I'll post a snippet.
Alternative Solutions
No response
Additional Context
No response
System Checks
- I do not believe that this feature can or should be implemented outside of Rocket.
- I was unable to find a previous request for this feature.