Skip to content

NetworkSegmentation: increase pod readiness timeout #29421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

NetworkSegmentation: increase pod readiness timeout #29421

wants to merge 1 commit into from

Conversation

kyrtapz
Copy link
Contributor

@kyrtapz kyrtapz commented Jan 10, 2025

No description provided.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 10, 2025
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 10, 2025

/test ?

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

@kyrtapz: The following commands are available to trigger required jobs:

/test e2e-aws-jenkins

/test e2e-aws-ovn-edge-zones

/test e2e-aws-ovn-fips

/test e2e-aws-ovn-image-registry

/test e2e-aws-ovn-microshift

/test e2e-aws-ovn-microshift-serial

/test e2e-aws-ovn-serial

/test e2e-gcp-ovn

/test e2e-gcp-ovn-builds

/test e2e-gcp-ovn-image-ecosystem

/test e2e-gcp-ovn-upgrade

/test e2e-metal-ipi-ovn-ipv6

/test images

/test lint

/test unit

/test verify

/test verify-deps

The following commands are available to trigger optional jobs:

/test 4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback

/test e2e-agnostic-ovn-cmd

/test e2e-aws

/test e2e-aws-csi

/test e2e-aws-disruptive

/test e2e-aws-etcd-certrotation

/test e2e-aws-etcd-recovery

/test e2e-aws-ovn

/test e2e-aws-ovn-cgroupsv2

/test e2e-aws-ovn-etcd-scaling

/test e2e-aws-ovn-ipsec-serial

/test e2e-aws-ovn-kube-apiserver-rollout

/test e2e-aws-ovn-kubevirt

/test e2e-aws-ovn-single-node

/test e2e-aws-ovn-single-node-serial

/test e2e-aws-ovn-single-node-techpreview

/test e2e-aws-ovn-single-node-techpreview-serial

/test e2e-aws-ovn-single-node-upgrade

/test e2e-aws-ovn-upgrade

/test e2e-aws-ovn-upgrade-rollback

/test e2e-aws-ovn-upi

/test e2e-aws-ovn-virt-techpreview

/test e2e-aws-proxy

/test e2e-azure

/test e2e-azure-ovn-etcd-scaling

/test e2e-azure-ovn-upgrade

/test e2e-baremetalds-kubevirt

/test e2e-external-aws

/test e2e-external-aws-ccm

/test e2e-external-vsphere-ccm

/test e2e-gcp-csi

/test e2e-gcp-disruptive

/test e2e-gcp-fips-serial

/test e2e-gcp-ovn-etcd-scaling

/test e2e-gcp-ovn-rt-upgrade

/test e2e-gcp-ovn-techpreview

/test e2e-gcp-ovn-techpreview-serial

/test e2e-hypershift-conformance

/test e2e-metal-ipi-ovn

/test e2e-metal-ipi-ovn-dualstack

/test e2e-metal-ipi-ovn-dualstack-local-gateway

/test e2e-metal-ipi-ovn-kube-apiserver-rollout

/test e2e-metal-ipi-serial

/test e2e-metal-ipi-serial-ovn-ipv6

/test e2e-metal-ipi-virtualmedia

/test e2e-metal-ovn-single-node-live-iso

/test e2e-metal-ovn-single-node-with-worker-live-iso

/test e2e-openstack-ovn

/test e2e-openstack-serial

/test e2e-vsphere

/test e2e-vsphere-ovn-dualstack-primaryv6

/test e2e-vsphere-ovn-etcd-scaling

/test okd-e2e-gcp

/test okd-scos-e2e-aws-ovn

/test okd-scos-images

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd

pull-ci-openshift-origin-master-e2e-aws-csi

pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2

pull-ci-openshift-origin-master-e2e-aws-ovn-edge-zones

pull-ci-openshift-origin-master-e2e-aws-ovn-fips

pull-ci-openshift-origin-master-e2e-aws-ovn-kube-apiserver-rollout

pull-ci-openshift-origin-master-e2e-aws-ovn-microshift

pull-ci-openshift-origin-master-e2e-aws-ovn-microshift-serial

pull-ci-openshift-origin-master-e2e-aws-ovn-serial

pull-ci-openshift-origin-master-e2e-aws-ovn-single-node

pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial

pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade

pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade

pull-ci-openshift-origin-master-e2e-gcp-csi

pull-ci-openshift-origin-master-e2e-gcp-ovn

pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade

pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade

pull-ci-openshift-origin-master-e2e-hypershift-conformance

pull-ci-openshift-origin-master-e2e-metal-ipi-ovn

pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6

pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-kube-apiserver-rollout

pull-ci-openshift-origin-master-e2e-openstack-ovn

pull-ci-openshift-origin-master-images

pull-ci-openshift-origin-master-lint

pull-ci-openshift-origin-master-okd-scos-e2e-aws-ovn

pull-ci-openshift-origin-master-unit

pull-ci-openshift-origin-master-verify

pull-ci-openshift-origin-master-verify-deps

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kyrtapz
Once this PR has been reviewed and has the lgtm label, please assign adambkaplan for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 10, 2025

/test e2e-aws-ovn-single-node-techpreview
/test e2e-gcp-ovn-techpreview
/test e2e-aws-ovn-virt-techpreview

@tssurya
Copy link
Contributor

tssurya commented Jan 10, 2025

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-dualstack-techpreview
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6-techpreview

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

@tssurya: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@tssurya
Copy link
Contributor

tssurya commented Jan 10, 2025

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-dualstack-techpreview 5
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

@tssurya: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-dualstack-techpreview
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/1b05a080-cf58-11ef-8081-4e6ec1d95c4a-0

@tssurya
Copy link
Contributor

tssurya commented Jan 10, 2025

/payload periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

@tssurya: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@tssurya
Copy link
Contributor

tssurya commented Jan 10, 2025

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 10, 2025

@tssurya: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/9dc1a500-cf58-11ef-9cc5-846ca90eafd4-0

@tssurya
Copy link
Contributor

tssurya commented Jan 11, 2025

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-29421-nightly-4.19-e2e-azure-ovn-runc-techpreview/1877712430579060736

yikes we are hitting other issues:

{  fail [github.com/openshift/origin/test/extended/networking/network_segmentation_endpointslice_mirror.go:171]: Expected success, but got an error:
    <*errors.joinError | 0xc001733ec0>: 
    error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkReady=True --timeout 5s:
    Command stdout:
    
    stderr:
    Error from server (NotFound): userdefinednetworks.k8s.ovn.org "gryffindor" not found
    
    error:
    exit status 1
    error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkCreated=True --timeout 5s:
    Command stdout:
    
    stderr:
    error: timed out waiting for the condition on userdefinednetworks/gryffindor
    
    error:
    exit status 1
    {
        errs: [
            <exec.CodeExitError>{
                Err: <*errors.errorString | 0xc001e76f60>{
                    s: "error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkReady=True --timeout 5s:\nCommand stdout:\n\nstderr:\nError from server (NotFound): userdefinednetworks.k8s.ovn.org \"gryffindor\" not found\n\nerror:\nexit status 1",
                },
                Code: 1,
            },
            <exec.CodeExitError>{
                Err: <*errors.errorString | 0xc001e36d50>{
                    s: "error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkCreated=True --timeout 5s:\nCommand stdout:\n\nstderr:\nerror: timed out waiting for the condition on userdefinednetworks/gryffindor\n\nerror:\nexit status 1",
                },
                Code: 1,
            },
        ],
    }
Ginkgo exit error 1: exit with code 1}

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 13, 2025 •
edited
Loading

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-29421-nightly-4.19-e2e-azure-ovn-runc-techpreview/1877712430579060736

yikes we are hitting other issues:

{  fail [github.com/openshift/origin/test/extended/networking/network_segmentation_endpointslice_mirror.go:171]: Expected success, but got an error:
    <*errors.joinError | 0xc001733ec0>: 
    error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkReady=True --timeout 5s:
    Command stdout:
    
    stderr:
    Error from server (NotFound): userdefinednetworks.k8s.ovn.org "gryffindor" not found
    
    error:
    exit status 1
    error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkCreated=True --timeout 5s:
    Command stdout:
    
    stderr:
    error: timed out waiting for the condition on userdefinednetworks/gryffindor
    
    error:
    exit status 1
    {
        errs: [
            <exec.CodeExitError>{
                Err: <*errors.errorString | 0xc001e76f60>{
                    s: "error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkReady=True --timeout 5s:\nCommand stdout:\n\nstderr:\nError from server (NotFound): userdefinednetworks.k8s.ovn.org \"gryffindor\" not found\n\nerror:\nexit status 1",
                },
                Code: 1,
            },
            <exec.CodeExitError>{
                Err: <*errors.errorString | 0xc001e36d50>{
                    s: "error running /usr/bin/kubectl --server=https://api.ci-op-c6v3rbgf-4414c.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/tmp/kubeconfig-2679390344 --namespace=e2e-test-endpointslices-mirror-e2e-mqq65 wait userdefinednetwork gryffindor --for condition=NetworkCreated=True --timeout 5s:\nCommand stdout:\n\nstderr:\nerror: timed out waiting for the condition on userdefinednetworks/gryffindor\n\nerror:\nexit status 1",
                },
                Code: 1,
            },
        ],
    }
Ginkgo exit error 1: exit with code 1}

This seems like a separate flake, I am not seeing the timeout issue in any of the runs which is good.

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 13, 2025

/test e2e-gcp-ovn-techpreview

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 13, 2025

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 13, 2025

@kyrtapz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f5749b00-d1aa-11ef-96f8-4170f9f5296a-0

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 14, 2025

/test e2e-gcp-ovn-techpreview
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 14, 2025

@kyrtapz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/61be3a70-d253-11ef-9b67-7d3714595219-0

@kyrtapz
Copy link
Contributor Author

kyrtapz commented Jan 15, 2025

/test e2e-gcp-ovn-techpreview
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 15, 2025

@kyrtapz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-azure-ovn-runc-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/ed62c9a0-d330-11ef-9c0a-76b20bbe94ed-0

@tssurya
Copy link
Contributor

tssurya commented Jan 15, 2025

pulled this commit into #29428 let's see if it helps

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 12, 2025
@openshift-merge-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 2, 2025

@kyrtapz: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/verify 6993b7c link true /test verify
ci/prow/e2e-aws-ovn-edge-zones 6993b7c link true /test e2e-aws-ovn-edge-zones
ci/prow/e2e-vsphere-ovn 6993b7c link true /test e2e-vsphere-ovn
ci/prow/images 6993b7c link true /test images
ci/prow/e2e-aws-ovn-microshift-serial 6993b7c link true /test e2e-aws-ovn-microshift-serial
ci/prow/e2e-vsphere-ovn-upi 6993b7c link true /test e2e-vsphere-ovn-upi
ci/prow/e2e-metal-ipi-ovn-ipv6 6993b7c link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-gcp-ovn-upgrade 6993b7c link true /test e2e-gcp-ovn-upgrade
ci/prow/e2e-aws-ovn-microshift 6993b7c link true /test e2e-aws-ovn-microshift
ci/prow/lint 6993b7c link true /test lint
ci/prow/verify-deps 6993b7c link true /test verify-deps
ci/prow/e2e-aws-ovn-fips 6993b7c link true /test e2e-aws-ovn-fips
ci/prow/okd-scos-images 6993b7c link true /test okd-scos-images
ci/prow/unit 6993b7c link true /test unit
ci/prow/e2e-gcp-ovn 6993b7c link true /test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-serial-2of2 6993b7c link true /test e2e-aws-ovn-serial-2of2
ci/prow/e2e-aws-ovn-serial-1of2 6993b7c link true /test e2e-aws-ovn-serial-1of2
ci/prow/e2e-aws-ovn-serial 6993b7c link true /test e2e-aws-ovn-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt openshift-trt
Copy link

openshift-trt bot commented May 2, 2025

Job Failure Risk Analysis for sha: 6993b7c

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-serial IncompleteTests
Tests for this run (2) are below the historical average (1134): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-1of2 IncompleteTests
Tests for this run (2) are below the historical average (1133): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-2of2 IncompleteTests
Tests for this run (2) are below the historical average (1055): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kyrtapz @tssurya @openshift-merge-robot