Debug ST pipelines being stuck on deletion by im-konge · Pull Request #12071 · strimzi/strimzi-kafka-operator

im-konge · 2025-10-24T17:27:51Z

Type of change

Select the type of your PR

Bugfix
Enhancement / new feature
Refactoring
Documentation

Description

Please describe your pull request

Checklist

Please go through this checklist and make sure all applicable tasks have been done

Write tests
Make sure all tests pass
Update documentation
Check RBAC rights for Kubernetes / OpenShift roles
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md
Supply screenshots for visual changes, such as Grafana dashboards

im-konge · 2025-10-24T17:30:28Z

/gha run pipeline=regression

codecov · 2025-10-24T18:16:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.79%. Comparing base (a030653) to head (5dcb468).
⚠️ Report is 10 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12071      +/-   ##
============================================
+ Coverage     74.77%   74.79%   +0.01%     
- Complexity     6611     6625      +14     
============================================
  Files           376      377       +1     
  Lines         25325    25359      +34     
  Branches       3389     3396       +7     
============================================
+ Hits          18937    18966      +29     
- Misses         5000     5005       +5     
  Partials       1388     1388

see 12 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-10-24T18:16:56Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-25T00:31:50Z

❌ System test verification failed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-29T10:01:11Z

/gha run pipeline=regression

github-actions · 2025-10-29T10:50:11Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-29T17:04:56Z

❌ System test verification failed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-29T21:34:12Z

/gha run pipeline=regression

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-29T22:46:43Z

/gha run pipeline=regression

github-actions · 2025-10-29T23:04:36Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-30T05:18:55Z

❌ System test verification failed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-30T11:41:04Z

/gha run pipeline=regression

github-actions · 2025-10-30T12:31:11Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-30T13:09:54Z

/gha run pipeline=regression

github-actions · 2025-10-30T13:59:01Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-30T17:01:36Z

🎉 System test verification passed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

github-actions · 2025-10-30T20:10:44Z

❌ System test verification failed: link

im-konge · 2025-10-30T20:18:29Z

/gha run pipeline=regression

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-30T21:25:07Z

/gha run pipeline=regression

github-actions · 2025-10-30T21:26:56Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

im-konge · 2025-10-31T06:50:49Z

/gha run pipeline=regression

github-actions · 2025-10-31T06:51:33Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-31T10:58:52Z

🎉 System test verification passed: link

github-actions · 2025-10-31T13:19:24Z

❌ System test verification failed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-10-31T18:00:47Z

/gha run pipeline=regression

github-actions · 2025-10-31T18:01:42Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-10-31T22:57:14Z

❌ System test verification failed: link

im-konge · 2025-10-31T23:05:50Z

/gha run pipeline=regression

github-actions · 2025-10-31T23:06:21Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-11-01T03:34:16Z

🎉 System test verification passed: link

im-konge · 2025-11-01T11:06:42Z

/gha run pipeline=regression

github-actions · 2025-11-01T11:07:12Z

⏳ System test verification started: link

The following 6 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)

Tests will start after successful build completion.

github-actions · 2025-11-01T15:55:38Z

❌ System test verification failed: link

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge · 2025-11-03T11:54:26Z

The main reason why jobs were stuck is fixed, so I'm closing this PR as it is not needed anymore.

im-konge added 2 commits October 27, 2025 12:47

Change log level to trace

1f9eba5

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

fabric8 to debug

1bfbf19

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge force-pushed the debug-jobs-stuck branch from 067e77b to 1bfbf19 Compare October 29, 2025 10:00

try with trace

febf482

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

delete kafka background

6c88e9d

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

build test-frame step

8075521

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge force-pushed the debug-jobs-stuck branch from f84d456 to 8075521 Compare October 30, 2025 11:40

move it to action

7a9ce3e

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

try to use different version of test-frame when building it in GHA

8a10248

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

comment out the tests part as we don't need it for debug

8f0adc8

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge mentioned this pull request Oct 31, 2025

[ST] Disable async delete to workaround jobs being stuck #12078

Closed

1 task

try to delete knp first

6065534

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

Revert debug stuff

5dcb468

Signed-off-by: Lukas Kral <lukywill16@gmail.com>

im-konge force-pushed the debug-jobs-stuck branch from 7c7b0f4 to 5dcb468 Compare November 3, 2025 11:44

im-konge closed this Nov 3, 2025

Conversation

im-konge commented Oct 24, 2025

Type of change

Description

Checklist

Uh oh!

im-konge commented Oct 24, 2025

Uh oh!

codecov Bot commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Oct 24, 2025

Uh oh!

github-actions Bot commented Oct 25, 2025

Uh oh!

im-konge commented Oct 29, 2025

Uh oh!

github-actions Bot commented Oct 29, 2025

Uh oh!

github-actions Bot commented Oct 29, 2025

Uh oh!

im-konge commented Oct 29, 2025

Uh oh!

im-konge commented Oct 29, 2025

Uh oh!

github-actions Bot commented Oct 29, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

im-konge commented Oct 30, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

im-konge commented Oct 30, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

im-konge commented Oct 30, 2025

Uh oh!

im-konge commented Oct 30, 2025

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

im-konge commented Oct 31, 2025

Uh oh!

github-actions Bot commented Oct 31, 2025

Uh oh!

github-actions Bot commented Oct 31, 2025

Uh oh!

github-actions Bot commented Oct 31, 2025

Uh oh!

im-konge commented Oct 31, 2025

Uh oh!

github-actions Bot commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2025

Uh oh!

im-konge commented Oct 31, 2025

Uh oh!

github-actions Bot commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Nov 1, 2025

Uh oh!

im-konge commented Nov 1, 2025

Uh oh!

github-actions Bot commented Nov 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Nov 1, 2025

Uh oh!

im-konge commented Nov 3, 2025

Uh oh!

Reviewers

codecov Bot commented Oct 24, 2025 •

edited

Loading

github-actions Bot commented Oct 31, 2025 •

edited

Loading

github-actions Bot commented Oct 31, 2025 •

edited

Loading

github-actions Bot commented Nov 1, 2025 •

edited

Loading