Skip to content

DNM: Use $PIP instead of explicit $SYSTEM_PIP #2907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

amartyasinha
Copy link
Contributor

@amartyasinha amartyasinha commented Apr 17, 2025

This patch is the first step in enforcing the use of venv for cifmw. We want that all deps of cifmw should be within it's venv and should not consume system's pip.

It is important as from python3.12 onwards, use of system wide packages are discouraged, and it is recommended to use venv.
Ref: https://issues.redhat.com/browse/OSPRH-15882

@amartyasinha amartyasinha requested a review from a team as a code owner April 17, 2025 09:41
Copy link
Contributor

openshift-ci bot commented Apr 17, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/c6df55d6647445d280386661c600e604

✔️ openstack-k8s-operators-content-provider SUCCESS in 38m 25s
podified-multinode-edpm-deployment-crc FAILURE in 17m 05s
cifmw-crc-podified-edpm-baremetal FAILURE in 23m 14s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 38s
✔️ cifmw-pod-pre-commit SUCCESS in 12m 34s
✔️ build-push-container-cifmw-client SUCCESS in 19m 36s

@amartyasinha
Copy link
Contributor Author

Closing this for now, as this change should be one of the last steps. Investigating how we can do that. Closing it for now.

@amartyasinha amartyasinha reopened this Apr 28, 2025
@amartyasinha amartyasinha changed the title Use $PIP instead of explicit $SYSTEM_PIP DNM: Use $PIP instead of explicit $SYSTEM_PIP Apr 28, 2025
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a571fc25cdea4f2db7d73f8832705c94

✔️ openstack-k8s-operators-content-provider SUCCESS in 34m 58s
podified-multinode-edpm-deployment-crc FAILURE in 19m 49s
cifmw-crc-podified-edpm-baremetal FAILURE in 19m 24s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 30s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 25s
✔️ build-push-container-cifmw-client SUCCESS in 19m 34s

…explicit $SYSTEM_PIP

This patch is the first step in enforcing the use of venv for cifmw. We want that all deps of cifmw should be within it's venv and should not consume system's pip.
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/b9122edaafc54a649664eb0a1b7a1b51

openstack-k8s-operators-content-provider FAILURE in 12m 45s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 35s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 38s
✔️ build-push-container-cifmw-client SUCCESS in 16m 39s

@amartyasinha
Copy link
Contributor Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/c6750bd2e6b24e83ae47d0390c7e3a52

openstack-k8s-operators-content-provider FAILURE in 12m 32s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 13s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 37s
✔️ build-push-container-cifmw-client SUCCESS in 15m 41s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/cd204735d6454fc1a1a9ce0e620ace66

✔️ openstack-k8s-operators-content-provider SUCCESS in 36m 42s
podified-multinode-edpm-deployment-crc FAILURE in 20m 48s
cifmw-crc-podified-edpm-baremetal FAILURE in 20m 18s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 18s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 55s
✔️ build-push-container-cifmw-client SUCCESS in 16m 37s

It may be possible that since we are not setting up USE_VENV to yes, it's consuming system env. This commit is just to test it.
It may be possible that zuul is not loading HOME var value. This commit is just to test it by explicitly setting it's value.
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/e45446f88a514e48b85bff206d88e7e5

✔️ openstack-k8s-operators-content-provider SUCCESS in 36m 17s
podified-multinode-edpm-deployment-crc FAILURE in 19m 53s
cifmw-crc-podified-edpm-baremetal FAILURE in 21m 10s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 17s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 18s
build-push-container-cifmw-client RETRY_LIMIT in 2m 29s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/6735ef02e8eb43119e2990169f9cbd4e

✔️ openstack-k8s-operators-content-provider SUCCESS in 45m 20s
podified-multinode-edpm-deployment-crc FAILURE in 18m 52s
cifmw-crc-podified-edpm-baremetal FAILURE in 28m 34s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 20s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 01s
build-push-container-cifmw-client RETRY_LIMIT in 2m 28s

Even if ansible-playbook is consuming venv, but ansible_python_interpreter is set to auto, it will depend on system to setup that, and that's why the modules were consuming /usr/bin/python3 instead of venv python. This patch should fix it.
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/50bf6139cfed44148342ee1f45e4c0f4

✔️ openstack-k8s-operators-content-provider SUCCESS in 36m 28s
podified-multinode-edpm-deployment-crc FAILURE in 20m 02s
cifmw-crc-podified-edpm-baremetal FAILURE in 22m 14s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 48s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 09s
build-push-container-cifmw-client RETRY_LIMIT in 2m 32s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ade752aee7f445c7834d539972a4327a

openstack-k8s-operators-content-provider RETRY_LIMIT in 2m 39s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 07s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 04s
build-push-container-cifmw-client RETRY_LIMIT in 1m 53s

libvirt package has host OS bindings, thus installing within venv is pain. Ref: ansible-collections/community.libvirt#71 (comment)

Thus, installing through distribution package (dnf), and including in venv through --system-site-packages
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/6204cb672bdc407ea4a0e355c881440e

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 09s
podified-multinode-edpm-deployment-crc FAILURE in 19m 31s
cifmw-crc-podified-edpm-baremetal POST_FAILURE in 1h 34m 12s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 00s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 40s
build-push-container-cifmw-client RETRY_LIMIT in 2m 30s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a5e8fb74c1a34f2fbfa3ee9d0fd7d7a3

✔️ openstack-k8s-operators-content-provider SUCCESS in 38m 39s
podified-multinode-edpm-deployment-crc FAILURE in 18m 40s
cifmw-crc-podified-edpm-baremetal FAILURE in 24m 49s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 53s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 15s
build-push-container-cifmw-client RETRY_LIMIT in 3m 10s

Instead of setting this var during runtime, it is better to set it up in config file.
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/76c183198c5641ea8bc0ce3bfef9858e

openstack-k8s-operators-content-provider FAILURE in 4m 17s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 35s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 03s
build-push-container-cifmw-client RETRY_LIMIT in 2m 33s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/161e5090f4624c5d81fae38f86e5d172

openstack-k8s-operators-content-provider FAILURE in 4m 16s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 02s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 06s
build-push-container-cifmw-client FAILURE in 3m 45s

in order to use venv python, we may want to ensure that interpreter_python is set to the venv's python whenever it exists
@danpawlik
Copy link
Contributor

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants