Component(s)
loki.source.syslog
What's wrong?
The UDP code for loki.source.syslog is written without considering that UDP is a connectionless protocol. In
internal/component/loki/source/syslog/internal/syslogtarget/transport.go, the acceptPackets function has the following code:
for {
...
n, addr, err = t.udpConn.ReadFrom(buf)
if n <= 0 && err != nil {
level.Warn(t.logger).Log("msg", "failed to read packets", "addr", addr, "err", err)
continue
}
stream, ok := streams[addr.String()]
if !ok {
stream = NewConnPipe(addr)
streams[addr.String()] = stream
t.openConnections.Add(1)
go t.handleRcv(stream)
}
}
handleRcv has the following loop:
for {
datagram := make([]byte, t.maxMessageLength())
n, err := c.Read(datagram)
if err != nil {
if err == io.EOF {
break
}
level.Warn(t.logger).Log("msg", "error reading from pipe", "err", err)
continue
}
...
}
Because UDP is connectionless, c.Read will never return EOF under normal circumstances. This means handleRcv is an infinite loop, so alloy will hold onto this goroutine, with datagram allocated with t.maxMessageLength(), forever.
Since UDP is connectionless, clients are likely to use different source ports to send syslog messages over time, meaning alloy will have hundreds to thousands of these handleRecv goroutines for each syslog client.
See attached pprof for evidence of memory leak
Steps to reproduce
- Start alloy with a loki.source.syslog component with a UDP listener.
- Send UDP syslog messages to the listener from a large number of IP:source port pairs
- Observe memory leak
System information
No response
Software version
No response
Configuration
Logs
Tip
React with 👍 if this issue is important to you.
Component(s)
loki.source.syslog
What's wrong?
The UDP code for loki.source.syslog is written without considering that UDP is a connectionless protocol. In
internal/component/loki/source/syslog/internal/syslogtarget/transport.go, the acceptPackets function has the following code:handleRcv has the following loop:
Because UDP is connectionless, c.Read will never return EOF under normal circumstances. This means handleRcv is an infinite loop, so alloy will hold onto this goroutine, with
datagramallocated with t.maxMessageLength(), forever.Since UDP is connectionless, clients are likely to use different source ports to send syslog messages over time, meaning alloy will have hundreds to thousands of these handleRecv goroutines for each syslog client.
See attached pprof for evidence of memory leak
Steps to reproduce
System information
No response
Software version
No response
Configuration
Logs
Tip
React with 👍 if this issue is important to you.