Handling configuration issuse and getting notifications about pool state at startup

[mcesaro]

This pull request add the possibility to define the max number of reconnection retries and then stop in case there are problems in the pool configuration or if the postgres database cannot be reached.
The caller process can receive notification messages when a db connection from the pool is available or if the pool has no connections to the database. See the readme file for the details.
If the connection cannot be established, the pool process is terminated, so it is possible to try again when the configuration/connectivity issues are removed.

[lpil]

Sounds cool! When would you use this new feature? Trying to fully understand the use cases. Thank you

[mcesaro]

I found the need when I tried to start an application and the postgres connection setup info was dynamically provided by a central configuration server and it was wrongly configured. Same also when postgres was stopped for maintenance at my application startup time. The pool retried indefinitely and my idea was to send a message to the operator that there is a problem with the postgres service. Also, fixing the confugration didn't work outright because the pool was already initialized and the only way to use the new config is by restarting it, stopping the whole application. With this PR, when I receive a notification that at least one connection with postgres is up, the rest of the application can act accordingly. If not, just the pool is killed so I can retry later.

[tsloughter]

Do you really want to notify for each connection that fails instead of the pool? But also, couldn't the process that wants to be notified instead monitor the pool process?

I figured the use of backoff made this feature unnecessary since it won't just keep constantly flooding with attempts to connect -- though the backoff does need to be made configurable as it right now will not go above trying to reconnect every 10 seconds.

I wonder if exposing a reconfigure or restart function would be cleaner.

I'm definitely for the idea of being able to kick the pool easily when you know it will never connect successfully and have a fixed configuration.

[mcesaro]

A single notification is enough, as the problem happens only at the pool creation time and we shouldn't care of a single connection down at runtime.
However, if all the connections in the pool fail (for example because the db server or the load balancer fail), it would be good to get a notification and do something accordingly.
The backoff delay works well if the database will be available sooner or later, but in my case the configuration of the pool is provided by a third party that might give a wrong IP address for the database or a bad user/password.
In this situation, the backoff method just retries indefinitely with the wrong parameters and you can't fix it but kill the pool and create a new pool.
Until the caller process receives a connection up notification, it can just avoid to use postgres.
I currently use the notification failure to log an error and give the operator the opportunity to fix the configuration and restart the connections pool.
A restart function would definitely a be cleaner solution.