Vector Source/Sink Question using NLB

[satellite-no]

I have 5 nodes sending traffic to 2 nodes using the Vector sink and Vector source. The 2 nodes are behind an AWS NLB which uses a hash flow hash algorithm based on the protocol, source IP address, source port, destination IP address, destination port, and TCP sequence number. The TCP connections from a client have different source ports and sequence numbers, and can be routed to different targets. Each individual TCP connection is routed to a single target for the life of the connection.

My question:
With this setup why would my traffic always be routing to 1 of the two nodes behind the NLB. I can stop vector and traffic immediately swaps to the other node so I know all is working and healthy. Based on the NLB hash flow hash algorithm is the vector source / sink doing something to keep the connection alive?

[satellite-no]

@pront any thoughts on this?

[pront]

My question:
With this setup why would my traffic always be routing to 1 of the two nodes behind the NLB. I can stop vector and traffic immediately swaps to the other node so I know all is working and healthy. Based on the NLB hash flow hash algorithm is the vector source / sink doing something to keep the connection alive?

What's your config? It's most likely that the TCP connection is persistent.

[satellite-no]

Hey @pront I'd agree with the persistent part. Is there a way to make it not persistent?

Config is as follows:

Sink:

type: vector
inputs:
  - core_*
address: 'https://vectorcore.mycompany.com:6000'

goes to Source:

type: vector
version: "2"
address: "0.0.0.0:6000"

[pront]

Is there a way to make it not persistent?

No, Vector currently does not provide a configuration option to disable persistent TCP connections.

Some thoughts; you could introduce an external load balancer that creates new connections on your behalf (e.g., a sidecar proxy like Envoy). Alternatively, if you changed the client connection settings or the Vector ports then that should trigger a NLB re-route. I haven’t tested any of this myself, so some experimentation may be needed.

[satellite-no]

Thanks for the reply's after a bunch of testing different configurations in vector and AWS I have the happy balance of swapping the NLB for an ALB and following the discussion here to configure the ALB and the vector sink. Now the LB seems to be distributing the data between the nodes and happy.