Skip to content

ECONNRESET due to upstream timeout causes crash #5

Open
@ImranR-TI

Description

@ImranR-TI

I'm seeing the http-proxy-3 crash with ECONNRESET in a specific scenario (minimal reproducible example below).

Basically I have the following setup: [client] --> [HAProxy] --> [Node proxy] --> [backend]

  1. A backend socket server
  2. A Node proxy server powered by http-proxy-3
    • This has some conditional logic which ensures that only specific requests are forwarded to the backend - the rest are simply ignored (no response). So clients are left waiting forever.
  3. A second proxy (HAProxy) in front of the Node proxy, that has some default request timeout value
    • This ensures that requests left waiting forever for some response (that will never come) do eventually have to break the connection.
    • I'm using HAProxy but I think anything similar would work here.

The bug I'm seeing has to do with this external timeout - it causes the Node proxy to crash with ECONNRESET. The error itself is not a problem, we would expect to see an error in this kind of situation. The problem is that the error crashes the entire process despite the presence of a proxy.on('error') listener.

I know this is a weird use case (my code should ideally respond to "ignored" requests with a 404 or something instead of completely ignoring them) but it does still expose what I think is a bug.

Minimal code to reproduce the bug:

// proxy.js

const http = require('http')
const httpProxy = require('http-proxy-3')

const proxy = httpProxy.createProxyServer({})

proxy.on('error', (err, req, res) => {
  console.error('Proxy error:', err)
  res.end('Something went wrong.')
})

const server = http.createServer((req, res) => {
  const target = 'http://localhost:3004'
  proxy.web(req, res, { target })
})

server.on('upgrade', (req, socket, head) => {
  if (false) { // Real use case has some specific condition, where some requests would go through and others wouldn't
    // We just do 'if (false)' to demonstrate the bug
    proxy.ws(req, socket, head, { target: 'ws://localhost:3004' })
  }
})

const port = 8000
server.listen(port, () => {
  console.log(`Proxy server listening on port ${port}`)
})
# haproxy.cfg

global
        maxconn 100000
        user haproxy
        group haproxy
        daemon
        log /dev/log local0 debug

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL).
        ssl-default-bind-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL
        ssl-default-bind-options no-sslv3

defaults
        mode    http
        timeout connect 180000 # 3 minutes
        timeout client  180000
        timeout server  180000
        fullconn 10000
        maxconn   5000

frontend ft_http
        bind :80
        mode http
        default_backend bk_http

backend bk_http
    mode http
    balance roundrobin
    option forwardfor
    default-server inter 1s
    server proxy1 127.0.0.1:8000 weight 255 check

Note backend.js has been omitted since it doesn't really matter in this case whether or not a backend exists - we never proxy the request anyways.

I test this with the wscat command: wscat -c http://localhost:80/socket

It eventually crashes with this error: error: Unexpected server response: 502

Proxy logs:

Proxy server listening on port 8000
node:events:496
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:216:20)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:170:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read'
}

Node.js v22.14.0

Thanks for this rewrite BTW, the old one had a bad memory leak (among other issues).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions