integration-service update by rh-tap-build-team[bot] · Pull Request #12275 · redhat-appstudio/infra-deployments

rh-tap-build-team · 2026-06-08T10:49:02Z

Included PRs:

Changelog

1d2c687 test(STONEINTG-1561): increase GitLab reporter coverage - fullsend-code
374edbd fix: upgrade gitlab pkg and around - @kasemAlem
855e348 fix: replace slices.Contains with loops to satisfy govet inline checker - @kasemAlem
3d19241 chore: demote golang.org/x/exp to indirect after removing slices usage - @kasemAlem
2b9ea39 chore(deps): update konflux references - @red-hat-konflux[bot]
748c474 Merge pull request Remove unused imagestream #1588 from konflux-ci/agent/1578-gitlab-reporter-coverage - @jsztuka
0b5e7ce chore: remove hongweiliu17 account - Hongwei Liu

github-actions · 2026-06-08T10:49:42Z

Kustomize Render Diff

Comparing 8b5750ca1 → 50918485f

Component	Environment	Changes
`components/integration/development`	development	+2 -2
`components/integration/staging/base`	staging	+2 -2
`components/integration/staging/stone-stage-p01`	staging	+2 -2

Total: 3 components, +6 -6 lines

📋 Full diff available in the workflow summary and as a downloadable artifact.

konflux-ci-qe-bot · 2026-06-08T12:32:44Z

🤖 Pipeline Failure Analysis

Category: Infrastructure

The pipeline failed due to severe infrastructure instability within the OpenShift cluster, characterized by an unresponsive Kubernetes API server and widespread TLS communication failures that prevented core services from functioning.

📋 Technical Details

Immediate Cause

The Konflux installation step (appstudio-e2e-tests/konflux-ci-install-konflux) failed because the client was unable to establish a connection to the Kubernetes API server, evidenced by repeated "Unable to connect to the server: EOF" errors. This indicates that the API server was either unresponsive, unreachable, or experienced a sudden connection termination.

Contributing Factors

Multiple must-gather diagnostic steps (appstudio-e2e-tests/gather-audit-logs, appstudio-e2e-tests/gather-extra, appstudio-e2e-tests/gather-must-gather, appstudio-e2e-tests/redhat-appstudio-gather) subsequently failed due to context deadline exceeded and widespread remote error: tls: internal error when attempting to collect data from various cluster components (e.g., Kubelet, OpenShift Authentication API server, etcd) across different nodes. This points to a systemic issue with node health, network connectivity, or certificate configuration, severely degrading inter-component communication. Additionally, high disk utilization (>= 80%) was observed on a master node, which could have contributed to the overall cluster instability and API server unresponsiveness.

Impact

The inability to connect to the Kubernetes API server and the subsequent widespread communication failures across cluster components directly prevented the successful installation of Konflux. This halted the pipeline at an early stage, making it impossible to proceed with the e2e tests and indicating a severely unhealthy and dysfunctional cluster environment.

🔍 Evidence

appstudio-e2e-tests/gather-audit-logs

Category: infrastructure
Root Cause: The must-gather operation timed out because the gathering pod failed to start within the allotted time, likely due to resource contention or performance issues within the cluster. High disk utilization on a master node was observed, which could be a contributing factor to the overall cluster instability or slowness.

Logs:

artifacts/appstudio-e2e-tests/gather-audit-logs/build-log.txt

the server is currently unable to handle the request (get imagestreams.image.openshift.io must-gather)

artifacts/appstudio-e2e-tests/gather-audit-logs/build-log.txt

W0608 12:04:32.054927      50 mustgather.go:423] volume percentage greater than or equal to 80 might cause filling up the disk space and have an impact on other components running on master

artifacts/appstudio-e2e-tests/gather-audit-logs/build-log.txt

[must-gather-5bgpn] OUT 2026-06-08T12:14:32.537484761Z gather did not start: context deadline exceeded

artifacts/appstudio-e2e-tests/gather-audit-logs/build-log.txt

error: gather did not start for pod must-gather-5bgpn: context deadline exceeded

appstudio-e2e-tests/gather-extra

Category: infrastructure
Root Cause: The OpenShift cluster's infrastructure experienced widespread TLS communication failures when must-gather attempted to collect data from nodes and other cluster components, indicating an underlying issue with node health, network connectivity, or certificate configuration. This unresponsiveness ultimately caused the must-gather process to time out.

Logs:

artifacts/appstudio-e2e-tests/gather-extra/build-log.txt

error: inspection completed with the errors occurred while gathering data:
    [skipping gathering routes.route.openshift.io/oauth-openshift due to error: the server doesn't have a resource type "routes", skipping gathering namespaces/openshift-authentication due to error: one or more errors occurred while gathering pod-specific data for namespace: openshift-authentication

    [one or more errors occurred while gathering container data for pod oauth-openshift-66994c95dc-bts5r:

    [Get "https://10.0.16.103:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-bts5r/oauth-openshift?timestamps=true": remote error: tls: internal error, Get "https://10.0.16.103:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-bts5r/oauth-openshift?previous=true&timestamps=true": remote error: tls: internal error]

artifacts/appstudio-e2e-tests/gather-extra/build-log.txt

oc --insecure-skip-tls-verify get --request-timeout=20s --raw /api/v1/nodes/ip-10-0-93-133.us-west-2.compute.internal/proxy/debug/pprof/heap
Error from server (ServiceUnavailable): error trying to reach service: remote error: tls: internal error

artifacts/appstudio-e2e-tests/gather-extra/build-log.txt

[must-gather-szl8w] OUT 2026-06-08T12:04:13.958446381Z gather did not start: context deadline exceeded

artifacts/appstudio-e2e-tests/gather-extra/build-log.txt

error: gather did not start for pod must-gather-szl8w: context deadline exceeded

appstudio-e2e-tests/gather-must-gather

Category: infrastructure
Root Cause: The Kubernetes cluster infrastructure is in a degraded state, leading to widespread TLS communication failures when attempting to retrieve container logs from Kubelet endpoints. This core issue prevents diagnostic tools like must-gather from functioning, suggesting network or certificate problems across the cluster nodes.

Logs:

artifacts/appstudio-e2e-tests/build-log.txt

Error running must-gather collection:
    gather did not start for pod must-gather-snwsp: context deadline exceeded

artifacts/appstudio-e2e-tests/build-log.txt

error running backup collection: inspection completed with the errors occurred while gathering data:
    [skipping gathering routes.route.openshift.io/oauth-openshift due to error: the server doesn't have a resource type "routes", skipping gathering namespaces/openshift-authentication due to error: one or more errors occurred while gathering pod-specific data for namespace: openshift-authentication

    [one or more errors occurred while gathering container data for pod oauth-openshift-66994c95dc-bts5r:

    [Get "https://10.0.16.103:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-bts5r/oauth-openshift?previous=true&timestamps=true": remote error: tls: internal error, Get "https://10.0.16.103:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-bts5r/oauth-openshift?timestamps=true": remote error: tls: internal error], one or more errors occurred while gathering container data for pod oauth-openshift-66994c95dc-fxlh4:

    [Get "https://10.0.60.38:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-fxlh4/oauth-openshift?timestamps=true": remote error: tls: internal error, Get "https://10.0.60.38:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-fxlh4/oauth-openshift?previous=true&timestamps=true": remote error: tls: internal error], one or more errors occurred while gathering container data for pod oauth-openshift-66994c95dc-m7fph:

    [Get "https://10.0.83.185:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-m7fph/oauth-openshift?previous=true&timestamps=true": remote error: tls: internal error, Get "https://10.0.83.185:10250/containerLogs/openshift-authentication/oauth-openshift-66994c95dc-m7fph/oauth-openshift?timestamps=true": remote error: tls: internal error]]

appstudio-e2e-tests/konflux-ci-install-konflux

Category: infrastructure
Root Cause: The Konflux installation step failed because the client was repeatedly unable to connect to the server, indicated by "EOF" errors. This suggests a loss of network connectivity or an unresponsive API server in the target Kubernetes environment.

Logs:

artifacts/appstudio-e2e-tests/konflux-ci-install-konflux/build-log.txt

Unable to connect to the server: EOF

appstudio-e2e-tests/redhat-appstudio-gather

Category: infrastructure
Root Cause: The OpenShift cluster is in a severely unhealthy state, preventing the oc commands from listing many expected custom resources and failing to gather logs due to widespread remote error: tls: internal error during communication with various core OpenShift components across multiple nodes.

Logs: