feat(cleanup): enable Docker cleanup for all backends and simplify code

[Copilot]

Description

Implemented Docker resource cleanup for all backends (aws, gce, azure, docker, k8s-*, xcloud) to fix the issue where Docker containers were only cleaned when backend=docker, leaving containers uncleaned when tests run on AWS/Azure/GCE backends.

The implementation follows the plan documented in docs/plans/docker-cleanup-all-backends.md and completes all 4 phases:

1. Enable Docker cleanup for all backends (except k8s-local)
2. Remove unused builder_name parameter
3. Simplify to local Docker daemon scanning only
4. Remove obsolete builder scanning code

Changes

Phase 1: Enable Docker Cleanup for All Backends

• Changed condition from if cluster_backend in ("docker", "") to if not cluster_backend.startswith("k8s-local")
• Added explanatory comment about why all backends need Docker cleanup
• Improved logging: Changed from "Going to delete..." to "Cleaned Docker container/image: " for better clarity

Phase 2: Remove Unused Parameters

• Removed unused builder_name parameter from clean_resources_docker() function
• Updated call to list_resources_docker() to not pass unused parameters
• Added docstring clarifying local-only scanning behavior

Phase 3: Simplify list_resources_docker

• Removed builder_name and group_as_builder parameters from function signature
• Removed call to obsolete list_clients_docker()
• Now uses docker.from_env() directly to get local Docker client
• Simplified from ~60 lines with parallel builder scanning to ~50 lines of straightforward local scanning

Phase 4: Remove Obsolete Code

• Verified list_clients_docker() not used elsewhere in codebase
• Removed entire list_clients_docker() function (47 lines of obsolete builder scanning code)
• Removed SSH connection logic, SSHAgent usage, and ParallelObject usage for builders

Bug Fixes

• Updated unit_tests/test_clean_cloud_resources_func.py::CleanResourcesDockerTest::test_destroy to match the new simplified function signature
• Reverted unnecessary <untagged> string handling in image logging to match original implementation

Key Points:

• Static builders are no longer used - all builder scanning code has been removed
• Docker cleanup now runs for ALL backends (aws, gce, azure, docker, k8s-*, xcloud) except k8s-local
• Simplified approach: local Docker daemon only, no remote builder scanning
• Total code reduction: ~50 lines removed, cleaner and more maintainable

Backend Coverage (Implemented)

Backend	Docker Cleanup	Status
aws	✅ YES	Local daemon scanned
gce	✅ YES	Local daemon scanned
azure	✅ YES	Local daemon scanned
docker	✅ YES	Local daemon scanned
k8s-eks	✅ YES	Local daemon scanned
k8s-gke	✅ YES	Local daemon scanned
k8s-local	⏭️ SKIP	No cleanup needed
xcloud	✅ YES	Local daemon scanned

Files Modified:

• sdcm/utils/resources_cleanup.py - Updated cleanup condition, improved logging, simplified function signature
• sdcm/utils/common.py - Simplified list_resources_docker, removed list_clients_docker
• unit_tests/test_clean_cloud_resources_func.py - Updated test to match new function signature
• docs/plans/docker-cleanup-all-backends.md - Implementation plan (all phases marked complete)

Ref: SCT-99

Testing

• Python syntax validated
• All 4 phases implemented
• Unit test fixed to match new signature
• Image logging reverted to original approach
• Full unit test suite (pending CI)
• Manual integration test with backend=aws recommended
• Manual integration test with backend=docker recommended

PR pre-checks (self review)

• I added the relevant backport labels
• I didn't leave commented-out/debugging code

Reminders

• Add New configuration option and document them (in sdcm/sct_config.py)
• Add unit tests to cover my changes (under unit-test/ folder)
• Update the Readme/doc folder relevant to this change (if needed)

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[fruch]

@copilot but if the backend is AWS or Azure, the docker cleanup isn't gonna happen at all, also the scan of all of the builders is disabled recently, can you review this part again ?

[Copilot]

@copilot but if the backend is AWS or Azure, the docker cleanup isn't gonna happen at all, also the scan of all of the builders is disabled recently, can you review this part again ?

You're absolutely right - I need to correct my analysis.

Issue 1: Docker cleanup doesn't run for AWS/Azure backends
The condition if cluster_backend in ("docker", "") at line 127 means Docker cleanup only runs when:

• Backend is explicitly "docker", OR
• Backend is empty/unset

If backend is "aws" or "azure", Docker resources are NOT cleaned up. This is a problem because tests on AWS/Azure backends may still create local Docker containers (e.g., for stress tools, monitoring).

Issue 2: Builder scanning disabled in Jenkins
Line 548 has if builder_name != "local" and getpass.getuser() != "jenkins" - this means when running as the jenkins user (which is the case in CI), remote builder scanning is completely skipped, and only the local Docker daemon is scanned.

My documentation incorrectly stated these features work in all cases. I'll update the analysis and fix the documentation to reflect the actual behavior.

[fruch]

@copilot list_resources_docker argument builder_name is never used, we should remove this scan for builder code from it, it should be working only locally

also update all of the in the plan, and the need to enable docker local cleanup in every cleanup
and drop for now all of the other comments or changes from the code
after the plan is done, we'll continue to the code

[Copilot]

@copilot list_resources_docker argument builder_name is never used, we should remove this scan for builder code from it, it should be working only locally

also update all of...

Done. Reverted all code changes and rewrote the plan (commit a24d0a5).

Changes:

1. All code files reverted to original state (no comments/changes)
2. New plan focuses on:
   • Remove unused builder_name parameter from list_resources_docker
   • Simplify to local Docker daemon only (no builder scanning)
   • Enable Docker cleanup for ALL backends: change condition to if not cluster_backend.startswith("k8s-local")

Ready to proceed with code implementation once plan is approved.

[fruch]

@copilot now go an implement the whole plan

[Copilot]

@copilot now go an implement the whole plan

Implemented all 4 phases in commits 3005e2a and f5a792b.

Summary:

• Phase 1: Docker cleanup now runs for ALL backends (except k8s-local) with improved logging
• Phase 2: Removed unused builder_name parameter from clean_resources_docker
• Phase 3: Simplified list_resources_docker to use docker.from_env() directly
• Phase 4: Removed obsolete list_clients_docker function (47 lines)

Total: -50 lines of code, simplified logic, Docker cleanup now works for aws/azure/gce backends.

[fruch]

@copilot fix precommit issue, and this test was failing:

12:40:52  FAILED unit_tests/test_clean_cloud_resources_func.py::CleanResourcesDockerTest::test_destroy - AssertionError: expected call not found.

[fruch]

@copilot where this <untagged> is coming from ?

[Copilot]

Fixed in commit 2274704. The <untagged> string was an unnecessary addition I made. Reverted to the original approach of logging image.tags directly as-is, which correctly handles all cases including None or empty lists.

[Copilot]

@copilot fix precommit issue, and this test was failing:

12:40:52  FAILED unit_tests/test_clean_cloud_resources_func.py::CleanResourcesDockerTest::test_destroy - AssertionError...

Fixed in commit 5fe0454. Updated test to call list_resources_docker with only tags_dict parameter, removing the obsolete builder_name=None and group_as_builder=False arguments that were removed during the Docker cleanup simplification.