Ability to "close" sockets without dropping pending requests

[tayler-king]

To preface: assume no use of websockets, purely HTTP connections

I'm putting together an application that can handle automatic scaling of uWS server instances based on load. I'm binding each instance to the same port and letting SO_REUSEPORT do its thing.

As an example, one method of scaling servers down would be to choose the uWS instance with the least amount of pending requests. By pending request I mean a request that has been accepted by the server and is being processed by the application, i.e. it is non-complete and has not finished responding to the client. Once all pending requests have been completed the instance would terminate.

The issue with the above approach is that I cannot fathom of a way to cleanly handling blocking of new requests when a uWS instance has been marked to be terminated, whilst using SO_REUSEPORT.

Would the better option be to just run each server on its own port and have the load balancer handle failing over if a request is rejected with status 503 for example? I would like to avoid this as then I would have to deal with the dynamic port selection of servers and assigning them to the load balancer.

[e3dio]

I cannot fathom of a way to cleanly handling blocking of new requests

import { App, us_listen_socket_close } from 'uWebSockets.js';

let listenSocket;
const app = App().listen(port, s => listenSocket = s);

// block new requests
us_listen_socket_close(listenSocket);

[tayler-king]

Will this keep existing connections open?

[e3dio]

Yep

[tayler-king]

Hah, that was a massive oversight on my behalf then. Thanks!

[uNetworkingAB]

We could possibly consider something similar to maxLifetime where you can specify the maximum time a client can hold a http connection. We have it for websockets for this very reason but not for http

[e3dio]

@tayler-king Browsers keep connections open with Connection: Keep Alive with 10 second server-side HTTP_IDLE_TIMEOUT, us_listen_socket_close stops new browser connections but still allows new requests/data from same connection which resets the timeout, you can add code to your request/onData handlers to stop new/existing requests as needed when attempting shutdown, res.close() and res.end(body, true) force close connection, app.close() force closes all connections

[tayler-king]

Many thanks for the info :)