High Water Mark for Client

[razorfrost44]

In Node Net Sockets, the server can pass along a highWaterMark to set write and read sizes. A client can't do this. It creates an issue when attempting to match buffer sizes from one side to the other. A client creation only takes the default low 16MB size currently. Is there a reason we don't allow clients to set sizes and only have this option in the create server for a socket?

Thanks!

[sunnypatell]

You're correct that highWaterMark is configurable for server-side sockets but not explicitly exposed for client-side sockets in Node.js. This behavior is due to how net.Socket is designed: when used as a server socket (server.createServer), it's instantiated internally, allowing explicit stream configuration. However, when used as a client (net.connect() or new net.Socket()), it defaults to 16MB for writable streams.

Why Isn’t highWaterMark Exposed for Clients?

1. Default Buffer Size Optimization:

   • For clients, the default highWaterMark (16MB) is optimized for most use cases, reducing unnecessary configuration.
   • A lower size could lead to excessive backpressure, while a higher one could introduce memory overhead.

2. Readability vs. Writability in Streams:

   • Node.js streams separate readable and writable buffering. On the server, setting highWaterMark makes sense because multiple clients can connect simultaneously, requiring controlled memory usage.
   • On the client, it's assumed that the buffer size doesn’t need fine-tuning as it typically communicates with a single server.

Workarounds: How to Set highWaterMark for a Client

Although not explicitly configurable, you can override it when creating the stream manually:

const net = require('net');

const socket = new net.Socket({
  writableHighWaterMark: 1024 * 1024, // 1MB
  readableHighWaterMark: 1024 * 1024, // 1MB
});

socket.connect(8080, 'example.com', () => {
  console.log('Connected to server with custom buffer sizes');
});

Alternatively, if using net.connect(), you can pass a preconfigured stream:

const { connect } = require('net');
const { Duplex } = require('stream');

const socket = connect({ port: 8080, host: 'example.com' });

const duplexStream = new Duplex({
  write(chunk, encoding, callback) {
    socket.write(chunk, encoding, callback);
  },
  read(size) {
    const chunk = socket.read(size);
    if (chunk) this.push(chunk);
  },
  writableHighWaterMark: 1024 * 1024, // 1MB
  readableHighWaterMark: 1024 * 1024, // 1MB
});

socket.pipe(duplexStream).pipe(process.stdout);

Possible Feature Proposal:

Given that adjusting highWaterMark can be useful in high-performance scenarios, a proposal to expose explicit client-side control might be worth discussing. It could be beneficial for optimizing throughput in low-latency, high-bandwidth connections.