Keep-alive for serverless environments

[goncalo-oliveira]

When working with serverless database environments, it's easy for a connection to get dropped while waiting for osm2pgsql to read a file. When that happens, we'll see something like

[CRITICAL]: Exception during execution: SSL connection has been closed unexpectedly

One of such database providers is Neon

For the first issue related to the SSL connection being closed unexpectedly, this can occur when the application tries to reuse an idle connection after the Neon compute has been suspended due to inactivity. If you plan to leave a connection idle, it would be good to increment the auto-suspend timer to meet the duration needed, or disable it altogether. The default auto-suspend is 5 minutes, and looking at the endpoint activity, it is suspending roughly 5 minutes after opening.

Naturally it's possible to workaround this by changing the default inactivity timeout to a bigger value on Neon compute settings. However, it would be nice if there was some sort of keep-alive mechanism to prevent the timeout from triggering, even if an additional option like --keep-alive for example, was necessary.

[joto]

Isn't this the same issue as #2290?

And then: If you are running osm2pgsql and the database is idle for 5 minutes than something is probably wrong with that setup. Osm2pgsql should keep the database busy more or less all the time.

[pnorman]

A flex table might go idle for 5 minutes if it has activity at the start and very end of processing. I'm not sure if that's very realistic. In any case, this can be done with PGOPTIONS right now.