Unable to setup Grpc keep alives #14045

ABAG603 · 2025-10-20T14:06:28Z

ABAG603
Oct 20, 2025

Hi,

I'm trying to setup Grpc keep alives between 2 collectors but behavior is quite strange to me. Can someone please help to understand why is that and maybe suggest a working config?

Client config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:

exporters:
  nop:
  otlp:
    endpoint: otel-receiver:4317
    timeout: 30s
    tls:
      insecure: true
    keepalive:
      time: 60s
      timeout: 5s
      permit_without_stream: true

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [] 
      exporters: [nop]
      
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlp]
      
    metrics:
      receivers: [otlp]
      processors: [] 
      exporters: [nop]

Server config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        keepalive:
          enforcement_policy:
            min_time: 30s
            permit_without_stream: true
      http:
        endpoint: 0.0.0.0:4318

processors:

exporters:
  nop:
  debug:

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [] 
      exporters: [nop]
      
    traces:
      receivers: [otlp]
      processors: [] 
      exporters: [debug]
      
    metrics:
      receivers: [otlp]
      processors: []
      exporters: [nop]

And here's the dump file (I kept only keep alive entries there)

172.19.0.4 is a client
172.19.0.2 is a server

   16 2025-10-20 13:39:43.392721   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=160 Ack=2155 Win=534 Len=0 TSval=1891153207 TSecr=3564076566
   17 2025-10-20 13:39:43.392726   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2155 Ack=161 Win=502 Len=0 TSval=3564091575 TSecr=1891138157
   18 2025-10-20 13:39:58.751639   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=160 Ack=2155 Win=534 Len=0 TSval=1891168566 TSecr=3564091575
   19 2025-10-20 13:39:58.751650   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2155 Ack=161 Win=502 Len=0 TSval=3564106934 TSecr=1891138157
   22 2025-10-20 13:40:14.112350   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=160 Ack=2155 Win=534 Len=0 TSval=1891183927 TSecr=3564106934
   23 2025-10-20 13:40:14.112361   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2155 Ack=161 Win=502 Len=0 TSval=3564122295 TSecr=1891138157
   29 2025-10-20 13:40:43.806449   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=177 Ack=2172 Win=534 Len=0 TSval=1891213621 TSecr=3564136535
   30 2025-10-20 13:40:43.806458   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2172 Ack=178 Win=502 Len=0 TSval=3564151989 TSecr=1891198167
   31 2025-10-20 13:40:59.170824   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=177 Ack=2172 Win=534 Len=0 TSval=1891228985 TSecr=3564151989
   32 2025-10-20 13:40:59.170866   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2172 Ack=178 Win=502 Len=0 TSval=3564167353 TSecr=1891198167
   33 2025-10-20 13:41:14.531469   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=177 Ack=2172 Win=534 Len=0 TSval=1891244346 TSecr=3564167353
   34 2025-10-20 13:41:14.531503   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2172 Ack=178 Win=502 Len=0 TSval=3564182714 TSecr=1891198167
   42 2025-10-20 13:41:43.711003   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=194 Ack=2189 Win=534 Len=0 TSval=1891273525 TSecr=3564196544
   43 2025-10-20 13:41:43.711043   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2189 Ack=195 Win=502 Len=0 TSval=3564211894 TSecr=1891258176
   44 2025-10-20 13:41:59.072545   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=194 Ack=2189 Win=534 Len=0 TSval=1891288887 TSecr=3564211894
   45 2025-10-20 13:41:59.072579   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2189 Ack=195 Win=502 Len=0 TSval=3564227255 TSecr=1891258176
   46 2025-10-20 13:42:14.431068   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=194 Ack=2189 Win=534 Len=0 TSval=1891304245 TSecr=3564227255
   47 2025-10-20 13:42:14.431100   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2189 Ack=195 Win=502 Len=0 TSval=3564242613 TSecr=1891258176
   55 2025-10-20 13:42:43.619080   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=211 Ack=2206 Win=534 Len=0 TSval=1891333434 TSecr=3564256553
   56 2025-10-20 13:42:43.619097   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2206 Ack=212 Win=502 Len=0 TSval=3564271802 TSecr=1891318185
   57 2025-10-20 13:42:58.974713   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=211 Ack=2206 Win=534 Len=0 TSval=1891348789 TSecr=3564271802
   58 2025-10-20 13:42:58.974747   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2206 Ack=212 Win=502 Len=0 TSval=3564287157 TSecr=1891318185
   59 2025-10-20 13:43:14.335596   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=211 Ack=2206 Win=534 Len=0 TSval=1891364150 TSecr=3564287157
   60 2025-10-20 13:43:14.335636   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2206 Ack=212 Win=502 Len=0 TSval=3564302518 TSecr=1891318185
   68 2025-10-20 13:43:43.526997   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=228 Ack=2223 Win=534 Len=0 TSval=1891393341 TSecr=3564316560
   69 2025-10-20 13:43:43.527004   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2223 Ack=229 Win=502 Len=0 TSval=3564331709 TSecr=1891378192
   70 2025-10-20 13:43:58.888379   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=228 Ack=2223 Win=534 Len=0 TSval=1891408703 TSecr=3564331709
   71 2025-10-20 13:43:58.888406   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2223 Ack=229 Win=502 Len=0 TSval=3564347071 TSecr=1891378192
   72 2025-10-20 13:44:14.246408   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=228 Ack=2223 Win=534 Len=0 TSval=1891424061 TSecr=3564347071
   73 2025-10-20 13:44:14.246434   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2223 Ack=229 Win=502 Len=0 TSval=3564362429 TSecr=1891378192
   81 2025-10-20 13:44:43.422259   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=245 Ack=2240 Win=534 Len=0 TSval=1891453237 TSecr=3564376574
   82 2025-10-20 13:44:43.422278   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2240 Ack=246 Win=502 Len=0 TSval=3564391605 TSecr=1891438206
   83 2025-10-20 13:44:58.782826   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=245 Ack=2240 Win=534 Len=0 TSval=1891468597 TSecr=3564391605
   84 2025-10-20 13:44:58.782848   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2240 Ack=246 Win=502 Len=0 TSval=3564406965 TSecr=1891438206
   85 2025-10-20 13:45:14.143376   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=245 Ack=2240 Win=534 Len=0 TSval=1891483957 TSecr=3564406965
   86 2025-10-20 13:45:14.143414   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2240 Ack=246 Win=502 Len=0 TSval=3564422326 TSecr=1891438206
   94 2025-10-20 13:45:43.839116   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=262 Ack=2257 Win=534 Len=0 TSval=1891513653 TSecr=3564436582
   95 2025-10-20 13:45:43.839143   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2257 Ack=263 Win=502 Len=0 TSval=3564452021 TSecr=1891498214
   96 2025-10-20 13:45:59.202689   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=262 Ack=2257 Win=534 Len=0 TSval=1891529016 TSecr=3564452021
   97 2025-10-20 13:45:59.202717   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2257 Ack=263 Win=502 Len=0 TSval=3564467384 TSecr=1891498214
   98 2025-10-20 13:46:14.566483   172.19.0.2 → 172.19.0.4   TCP 66 [TCP Keep-Alive] 4317 → 33156 [ACK] Seq=262 Ack=2257 Win=534 Len=0 TSval=1891544380 TSecr=3564467384
   99 2025-10-20 13:46:14.566546   172.19.0.4 → 172.19.0.2   TCP 66 [TCP Keep-Alive ACK] 33156 → 4317 [ACK] Seq=2257 Ack=263 Win=502 Len=0 TSval=3564482748 TSecr=1891498214

Few questions:

Why keep alive is initiated always by the server and not by the client?
Why are they sometimes sent with 15s interval and sometimes with 30s interval? I'd expect 1 minute interval as I configured for the client.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unable to setup Grpc keep alives #14045

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Unable to setup Grpc keep alives #14045

Uh oh!

ABAG603 Oct 20, 2025

Replies: 0 comments

ABAG603
Oct 20, 2025