-
Notifications
You must be signed in to change notification settings - Fork 104
improvement(images): drop supervisor from scylla SCT image #10627
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@fruch This is a draft. Can you take a look and suggest if the direction is OK and we can continue with it? (a run of artifacts-docker-test with this change - https://argus.scylladb.com/tests/scylla-cluster-tests/45752905-3c3e-4ab4-bda9-621ec5f220b6) For now this change manages the processes directly, but nothing is dropped from the scylla-sct image (supervisor is just ignored).
|
ab44553
to
b9f99c1
Compare
b9f99c1
to
da218d0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Drops the use of Supervisor in the Scylla SCT Docker image and adds direct Docker-based service management.
- Introduces
DockerCmdRunner
for executing commands and transferring files via Docker API. - Adds
DockerServiceManager
to start/stop/restart services without Supervisor. - Updates
cluster_docker.py
and tests to removesupervisorctl
calls and adapt to the new runner and manager.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
File | Description |
---|---|
unit_tests/test_docker_cmd_runner.py | Adds unit tests for the new DockerCmdRunner functionality. |
sdcm/utils/docker_service_manager.py | Implements DockerServiceManager for container-based services. |
sdcm/remote/docker_cmd_runner.py | Defines DockerCmdRunner with exec, tar create/extract, send/receive. |
sdcm/cluster_docker.py | Replaces supervisor calls with direct container operations and integrates DockerCmdRunner . |
artifacts_test.py | Skips supervisor-specific checks when using Docker backend. |
Comments suppressed due to low confidence (1)
sdcm/cluster_docker.py:273
- The
build_container_image
step was removed, sorun_container
may fail if the custom image isn't pre-built; consider restoring image build or ensuring the image exists before running.
ContainerManager.run_container(
a13d309
to
f6c3682
Compare
f6c3682
to
08b2eb7
Compare
I've tried
did you try with some other test? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about logging (e.g. syslog-ng)? also we no longer ship iproute
or rsync
pkg which might be needed by some nemesis.
Did you try running with podman?
yes, @fruch raised this concern that without node-exporter in the image, we won't have metrics, etc.
good question
And this is for comparison what we have for pr-provision test:
So without the dedicated scylla-sct image and its tooling we lack some logs. |
For logging in docker backend we never used syslong, so nothing new here As for metrics, that an issue for scylla core, of how gonna be building the docker image, we'll need a confirmation from that exporter is gonna be removed, and we'll see how to approach it. Even it mean that in the meanwhile docker backend won't gonna have (some) monitoring data |
I would recommend to rebase, and give CI one more run |
596d6ed
to
4474a84
Compare
The change drops rebuilding scylladb image as `scylla-sct` image. Instead, scylladb docker image is used as is for the docker backend. For that purpose the new DockerCmdRunner abstraction was introduced, to allow operating docker based nodes directly using docker API (instead of sshing into container) Additionally, artifacts_test was updated to skip subtests/operations for the services, which are no longer supported by scylladb docker image (or are about to be dropped) - node-exporter and scylla-hosekeeping .
4474a84
to
1855f49
Compare
@fruch scylla-cluster-tests/sdcm/cluster.py Line 1776 in 5dd5e9a
|
raise issue for that, we should be able to change |
Created #11230 task for this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove docker cli installation from any docker instance
@dimakr |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The change drops rebuilding scylladb image as
scylla-sct
image. Instead, scylladb docker image is used as is for the docker backend.For that purpose the new DockerCmdRunner abstraction was introduced, to allow operating docker based nodes directly using docker API (instead of sshing
into container).
Additionally, artifacts_test was updated to skip subtests/operations for the services, which are no longer supported by scylladb docker image (or are about
to be dropped) - node-exporter and scylla-hosekeeping .
Testing
The test was executed against the custom built image (built in the context of work in dist/docker: drop supervisor from docker image scylladb#22883), that has supervisor dropped
PR-provision-test-docker
test, which was executed locally to use the custom built image:PR pre-checks (self review)
backport
labelsReminders
sdcm/sct_config.py
)unit-test/
folder)