Commit 6c9ccad
committed
gha: capture per-pod diagnostics for sig-network conntrack failures
The sig-network Conntrack "should be able to preserve UDP traffic ...
NodePort" spec has been failing at conntrack.go:214 "Failed to connect to
backend 2" while the backend-1 check passed, which means Cilium already
forwarded and replied to the UDP NodePort flow correctly. The real trigger is
the client probe pod (pod-client) sitting in CrashLoopBackOff during the
one-minute backend-2 poll window, so its logs freeze and the substring poll
deterministically times out. We could not prove that from CI: the run uses
--disable-log-dump=true and the test namespace is torn down in AfterEach, so
pod-client's lastState.terminated exit code and reason, and the previous
container stdout that would show whether the nc loop ran to completion, were
gone by the time we looked.
This adds instrumentation instead of a behavioural change, because a genuine
backend-2 datapath regression would surface at the same line and masking it is
unacceptable. I pass --delete-namespace-on-failure=false to the ginkgo run so
only failed specs' namespaces survive past AfterEach, and in the post-test
gathering step, before the kind export, I capture pods -o wide and -o yaml,
sorted events, and for every pod left in a conntrack/services/nettest/network
namespace a describe plus current and --previous container logs. The next
occurrence will show pod-client's exit code and whether the 3000-iteration nc
loop completed, which is what tells a client-pod crash apart from a datapath
fault.
AIL:3
Signed-off-by: André Martins <andre@cilium.io>1 parent 2deb5d2 commit 6c9ccad
1 file changed
Lines changed: 34 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
296 | 296 | | |
297 | 297 | | |
298 | 298 | | |
299 | | - | |
| 299 | + | |
| 300 | + | |
300 | 301 | | |
301 | 302 | | |
302 | 303 | | |
| |||
349 | 350 | | |
350 | 351 | | |
351 | 352 | | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
352 | 385 | | |
353 | 386 | | |
354 | 387 | | |
| |||
0 commit comments