-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
Observed behavior
NATS Server 2.12.2 fails to accept connections when both proxy_protocol: true and tls {} are configured together. The server attempts to start the TLS handshake before consuming the PROXY protocol header, causing the connection to fail immediately.
Expected behavior
When proxy_protocol: true is enabled with TLS:
- NATS should read and parse PROXY protocol header first (plaintext)
- Then initiate TLS handshake with client
- Then proceed with normal NATS protocol negotiation
This is the standard sequence for PROXY protocol + TLS as implemented by other servers (HAProxy, nginx, etc.).
Additional context
- Without
proxy_protocol: true, HAProxy passthrough works perfectly with TLS - With
proxy_protocol: truebut WITHOUT TLS, connections likely work (not tested as we require mTLS) - NATS seems to ignore the PROXY protocol header entirely when TLS is configured
- This affects both PROXY protocol v1 (text) and v2 (binary)
- Related PR: (2.12) Add opt-in support for the PROXY protocol #7456 (PROXY protocol support added in 2.12.2)
Request
NATS should handle the PROXY protocol header before initiating the TLS handshake when both features are enabled together. This is critical for obtaining client source IP in auth callouts behind load balancers.
Use case
We use AWS Network Load Balancer with PROXY protocol to preserve client source IP for authorization validation in the auth callout service. Without this, we cannot distinguish between edge devices and internal services for security policies.
Server and client version
- NATS Server: v2.12.2-linux-arm64 (also tested with nats:2.12.2-alpine Docker image)
- Proxy: HAProxy 2.9-alpine
- Client: nats.py v2.11.0
Host environment
- OS: Linux (Docker containers)
- Architecture: ARM64 and x86_64 (tested both)
- Network: Docker bridge network, HAProxy proxying to NATS
Steps to reproduce
1. NATS Server Configuration (nats-server.conf):
port: 4222
server_name: "NATS-SERVER-01"
Enable PROXY protocol
proxy_protocol: true
TLS configuration
tls {
cert_file: "/etc/nats/certs/nats-server.pem"
key_file: "/etc/nats/certs/nats-server.key"
ca_file: "/etc/nats/certs/ca.pem"
verify: true
verify_and_map: true
timeout: 2
}2. HAProxy Configuration:
PROXY protocol v1
backend nats_backend
mode tcp
server nats1 nats-server:4222 send-proxy check
Also tested with PROXY protocol v2
server nats1 nats-server:4222 send-proxy-v2 check3. Client connects to HAProxy:
import nats
import ssl
ssl_ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
ssl_ctx.load_verify_locations(ca_cert)
ssl_ctx.load_cert_chain(certfile=client_cert, keyfile=client_key)
nc = await nats.connect(
servers=["tls://haproxy:4223"],
tls=ssl_ctx
)4. Observed connection sequence:
- HAProxy receives TLS connection from client
- HAProxy forwards connection to NATS with PROXY header:
PROXY TCP4 192.168.1.100 172.21.0.2 50000 4222\r\n - NATS creates connection:
[DBG] 172.21.0.4:41502 - cid:73 - Client connection created - NATS immediately attempts TLS handshake:
[DBG] 172.21.0.4:41502 - cid:73 - Starting TLS client connection handshake - NATS fails because it reads PROXY header as TLS:
[ERR] tls: first record does not look like a TLS handshake - Connection closed:
[DBG] Client connection closed: TLS Handshake Failure