flake: Functest — Functest

[robot-helena]

Detected by the QE bot on #472 (PR #472).

Check: Functest
Test id: (none extracted)
Reason: Cluster infrastructure failure: K8s API server connection refused (127.0.0.1:46229) and container runtime exit 125 errors indicate cluster went down mid-test, unrelated to PR's OVSDB mirror filtering logic

Command: make cluster-down 2>/dev/null; make cluster-up && make cluster-sync && make E2E_TEST_ARGS="-ginkgo.v -test.v -test.timeout 20m -ginkgo.noColor " functest; rc=$?; make cluster-down; exit $rc (exit 2)

Evidence

=== RUN   TestPlugin
Running Suite: Plugin Suite - /tmp/qe-pr-472-wt/tests
=====================================================
Random Seed: 1775889480

Will run 14 of 14 specs
------------------------------
[BeforeSuite] 
/tmp/qe-pr-472-wt/tests/tests_suite_test.go:42
  STEP: Waiting for namespace test-namespace to be removed, this can take a while ... @ 04/11/26 06:38:00.617
  STEP: Creating test-namespace namespace @ 04/11/26 06:38:00.624
[BeforeSuite] PASSED [0.023 seconds]
------------------------------
ovs-mirror 1.0.0 ovs traffic mirroring tests when an OVS bridge is configured on a node and both consumer and producer network attachment definitions are defined with a mirror configuration and 3 pods (2 producers and 1 consumer) are connected through it consumer pod should be able to monitor network traffic between producer pods
/tmp/qe-pr-472-wt/tests/mirror_test.go:91
  [... test steps ...]
• [52.835 seconds]
------------------------------
ovs-mirror 0.4.0 ovs traffic mirroring tests when an OVS bridge is configured on a node and both consumer and producer network attachment definitions are defined with a mirror configuration and 3 pods (2 producers and 1 consumer) are connected through it consumer pod should be able to monitor network traffic between producer pods
/tmp/qe-pr-472-wt/tests/mirror_test.go:91
  [... test steps ...]
• [50.750 seconds]
------------------------------
ovs-mirror 0.3.0 ovs traffic mirroring tests when an OVS bridge is configured on a node and both consumer and producer network attachment definitions are defined with a mirror configuration and 3 pods (2 producers and 1 consumer) are connected through it consumer pod should be able to monitor network traffic between producer pods
/tmp/qe-pr-472-wt/tests/mirror_test.go:91
  STEP: Adding ovs-bridge on the node @ 04/11/26 06:39:44.225
  STEP: Adding NetworkAttachmentDefinition ovs-net-prod of ovs bridge @ 04/11/26 06:39:44.413
  STEP: Adding NetworkAttachmentDefinition ovs-net-cons of ovs bridge @ 04/11/26 06:39:44.42
  STEP: Creating pod pod-cons with priviliged premission and ip 10.1.0.1/24 @ 04/11/26 06:39:44.437
  STEP: Waiting for pod container to be in Running state @ 04/11/26 06:39:44.445
  STEP: Creating pod pod-prod-1 with priviliged premission and ip 10.0.0.1/24 @ 04/11/26 06:39:49.502
  STEP: Waiting for pod container to be in Running state @ 04/11/26 06:39:49.51
  STEP: Creating pod pod-prod-2 with priviliged premission and ip 10.0.0.2/24 @ 04/11/26 06:39:54.537
  STEP: Waiting for pod container to be in Running state @ 04/11/26 06:39:54.553
  STEP: Running and parsing tcpdump result @ 04/11/26 06:39:59.586
  STEP: Pinging over the network @ 04/11/26 06:39:59.586
  STEP: Confirming that the communication was recorded @ 04/11/26 06:40:03.618
  STEP: Cleaning Pods in test-namespace namespace @ 04/11/26 06:40:03.65
  [FAILED] in [AfterEach] - /tmp/qe-pr-472-wt/tests/cluster/cluster.go:146 @ 04/11/26 06:40:57.605
  STEP: Cleaning NetworkAttachmentDefinition @ 04/11/26 06:40:57.605
  [FAILED] in [AfterEach] - /tmp/qe-pr-472-wt/tests/cluster/cluster.go:173 @ 04/11/26 06:40:57.605
  STEP: Removing ovs-bridge on the node @ 04/11/26 06:40:57.605
  [FAILED] in [AfterEach] - /tmp/qe-pr-472-wt/tests/node/node.go:61 @ 04/11/26 06:40:57.628
• [FAILED] [73.404 seconds]
ovs-mirror 0.3.0 ovs traffic mirroring tests when an OVS bridge is configured on a node and both consumer and producer network attachment definitions are defined with a mirror configuration and 3 pods (2 producers and 1 consumer) are connected through it [AfterEach] consumer pod should be able to monitor network traffic between producer pods
  [AfterEach] /tmp/qe-pr-472-wt/tests/mirror_test.go:87
  [It] /tmp/qe-pr-472-wt/tests/mirror_test.go:91

  [FAILED] Should succeed getting pod list in test namespace after deletion
  Unexpected error:
      <*rest.wrapPreviousError | 0x1fa7fd1426a0>: 
      Get "<localhost-url>": dial tcp 127.0.0.1:46229: connect: connection refused - error from a 
... [truncated]

Close this issue once the underlying flaky test is fixed.

[robot-helena]

Recurred on #481 (PR #481).

Evidence

hack/install-kind.sh /tmp/qe-pr-481-wt/build/_output/bin/
./cluster/down.sh
./cluster/up.sh
+++ realpath ./cluster/up.sh
++ dirname /tmp/qe-pr-481-wt/cluster/up.sh
+ SCRIPTS_PATH=/tmp/qe-pr-481-wt/cluster
+ source /tmp/qe-pr-481-wt/cluster/cluster.sh
++ export KIND_CLUSTER_NAME=ovs-cni
++ KIND_CLUSTER_NAME=ovs-cni
++ export KIND_NODE_IMAGE=localhost/kindest/node-ovs:latest
++ KIND_NODE_IMAGE=localhost/kindest/node-ovs:latest
+++ podman ps
+++ echo podman
++ OCI_BIN=podman
++ export OCI_BIN
++ '[' -z podman ']'
++ '[' podman = podman ']'
++ export KIND_EXPERIMENTAL_PROVIDER=podman
++ KIND_EXPERIMENTAL_PROVIDER=podman
++++ dirname /tmp/qe-pr-481-wt/cluster/cluster.sh
+++ cd /tmp/qe-pr-481-wt/cluster
+++ pwd
++ CLUSTER_SH_DIR=/tmp/qe-pr-481-wt/cluster
++ export CONTAINERS_CONF_OVERRIDE=/tmp/qe-pr-481-wt/cluster/podman-containers.conf
++ CONTAINERS_CONF_OVERRIDE=/tmp/qe-pr-481-wt/cluster/podman-containers.conf
+ echo 'Building custom kind node image with OVS'
Building custom kind node image with OVS
+ podman build -t localhost/kindest/node-ovs:latest /tmp/qe-pr-481-wt/cluster/kind-node/
STEP 1/2: FROM docker.io/kindest/node:v1.34.0
STEP 2/2: RUN apt-get update &&     apt-get install -y --no-install-recommends openvswitch-switch &&     apt-get clean &&     rm -rf /var/lib/apt/lists/* &&     systemctl enable openvswitch-switch
--> Using cache 9a7cedbd090585c5ee1f46fbfbf11da29bef8fa8f9d437f69924296610af5039
COMMIT localhost/kindest/node-ovs:latest
--> 9a7cedbd0905
Successfully tagged localhost/kindest/node-ovs:latest
Successfully tagged localhost/stress-test:latest
9a7cedbd090585c5ee1f46fbfbf11da29bef8fa8f9d437f69924296610af5039
+ echo 'Creating kind cluster'
Creating kind cluster
+++ cluster::kubeconfig
+++ echo -n /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
++ dirname /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
+ mkdir -p /tmp/qe-pr-481-wt/.kube
++ cluster::kubeconfig
++ echo -n /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
+ kind create cluster --name ovs-cni --image localhost/kindest/node-ovs:latest --config /tmp/qe-pr-481-wt/cluster/kind-config.yaml --kubeconfig /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "ovs-cni" ...
 • Ensuring node image (localhost/kindest/node-ovs:latest) 🖼  ...
 ✓ Ensuring node image (localhost/kindest/node-ovs:latest) 🖼
 • Preparing nodes 📦 📦   ...
 ✓ Preparing nodes 📦 📦 
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✓ Starting control-plane 🕹️
 • Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 • Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
 • Joining worker nodes 🚜  ...
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-ovs-cni"
You can now use your cluster with:

kubectl cluster-info --context kind-ovs-cni --kubeconfig /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni

Thanks for using kind! 😊
+ echo 'Waiting for nodes to be ready'
Waiting for nodes to be ready
+ ./cluster/kubectl.sh wait --for=condition=Ready nodes --all --timeout=300s
node/ovs-cni-control-plane condition met
node/ovs-cni-worker condition met
+ echo 'Starting Open vSwitch on nodes'
Starting Open vSwitch on nodes
++ ./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name
+ for n in $(./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name)
+ podman exec ovs-cni-control-plane systemctl start openvswitch-switch
+ for n in $(./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name)
+ podman exec ovs-cni-worker systemctl start openvswitch-switch
+ echo 'Deploying Multus'
Deploying Multus
+ MULTUS_VERSION=v4.0.1
+ MULTUS_MANIFEST=https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/v4.0.1/deployments/multus-daemonset.yml
+ curl -L https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/v4.0.1/deployments/multus-daemonset.yml
+ sed s/:snapshot/:v4.0.1/g
+ ./cluster/kubectl.sh apply -f -
  % 
... [truncated]

[robot-helena]

Recurred on #481 (PR #481).

Evidence

hack/install-kind.sh /tmp/qe-pr-481-wt/build/_output/bin/
./cluster/down.sh
./cluster/up.sh
+++ realpath ./cluster/up.sh
++ dirname /tmp/qe-pr-481-wt/cluster/up.sh
+ SCRIPTS_PATH=/tmp/qe-pr-481-wt/cluster
+ source /tmp/qe-pr-481-wt/cluster/cluster.sh
++ export KIND_CLUSTER_NAME=ovs-cni
++ KIND_CLUSTER_NAME=ovs-cni
++ export KIND_NODE_IMAGE=localhost/kindest/node-ovs:latest
++ KIND_NODE_IMAGE=localhost/kindest/node-ovs:latest
+++ podman ps
+++ echo podman
++ OCI_BIN=podman
++ export OCI_BIN
++ '[' -z podman ']'
++ '[' podman = podman ']'
++ export KIND_EXPERIMENTAL_PROVIDER=podman
++ KIND_EXPERIMENTAL_PROVIDER=podman
++++ dirname /tmp/qe-pr-481-wt/cluster/cluster.sh
+++ cd /tmp/qe-pr-481-wt/cluster
+++ pwd
++ CLUSTER_SH_DIR=/tmp/qe-pr-481-wt/cluster
++ export CONTAINERS_CONF_OVERRIDE=/tmp/qe-pr-481-wt/cluster/podman-containers.conf
++ CONTAINERS_CONF_OVERRIDE=/tmp/qe-pr-481-wt/cluster/podman-containers.conf
+ echo 'Building custom kind node image with OVS'
Building custom kind node image with OVS
+ podman build -t localhost/kindest/node-ovs:latest /tmp/qe-pr-481-wt/cluster/kind-node/
STEP 1/2: FROM docker.io/kindest/node:v1.34.0
STEP 2/2: RUN apt-get update &&     apt-get install -y --no-install-recommends openvswitch-switch &&     apt-get clean &&     rm -rf /var/lib/apt/lists/* &&     systemctl enable openvswitch-switch
--> Using cache 9a7cedbd090585c5ee1f46fbfbf11da29bef8fa8f9d437f69924296610af5039
COMMIT localhost/kindest/node-ovs:latest
--> 9a7cedbd0905
Successfully tagged localhost/kindest/node-ovs:latest
Successfully tagged localhost/stress-test:latest
9a7cedbd090585c5ee1f46fbfbf11da29bef8fa8f9d437f69924296610af5039
+ echo 'Creating kind cluster'
Creating kind cluster
+++ cluster::kubeconfig
+++ echo -n /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
++ dirname /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
+ mkdir -p /tmp/qe-pr-481-wt/.kube
++ cluster::kubeconfig
++ echo -n /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
+ kind create cluster --name ovs-cni --image localhost/kindest/node-ovs:latest --config /tmp/qe-pr-481-wt/cluster/kind-config.yaml --kubeconfig /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "ovs-cni" ...
 • Ensuring node image (localhost/kindest/node-ovs:latest) 🖼  ...
 ✓ Ensuring node image (localhost/kindest/node-ovs:latest) 🖼
 • Preparing nodes 📦 📦   ...
 ✓ Preparing nodes 📦 📦 
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✓ Starting control-plane 🕹️
 • Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 • Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
 • Joining worker nodes 🚜  ...
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-ovs-cni"
You can now use your cluster with:

kubectl cluster-info --context kind-ovs-cni --kubeconfig /tmp/qe-pr-481-wt/.kube/kind-config-ovs-cni

Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
+ echo 'Waiting for nodes to be ready'
Waiting for nodes to be ready
+ ./cluster/kubectl.sh wait --for=condition=Ready nodes --all --timeout=300s
node/ovs-cni-control-plane condition met
node/ovs-cni-worker condition met
+ echo 'Starting Open vSwitch on nodes'
Starting Open vSwitch on nodes
++ ./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name
+ for n in $(./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name)
+ podman exec ovs-cni-control-plane systemctl start openvswitch-switch
+ for n in $(./cluster/kubectl.sh get nodes --no-headers -o custom-columns=NAME:.metadata.name)
+ podman exec ovs-cni-worker systemctl start openvswitch-switch
+ echo 'Deploying Multus'
Deploying Multus
+ MULTUS_VERSION=v4.0.1
+ MULTUS_MANIFEST=https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/v4.0.1/deployments/multus-daemonset.yml
+ curl -L https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/v4.0.1/deployments/multus-daemonset.yml
+ se
... [truncated]