Allow pip packages to be optionally installed into a virtualenv

[jrosser]

pep-668 will force installation of python modules with pip into a virtualenv rather than the system python. This is already required for deban-12 and ubuntu-24.04.

This patch allows the caller of ansible-collection-kubernetes to pass the path to a pre-created virtualenv to install python target host requirements for ansible modules into.

[jrosser]

@mnaser i've got CI that exercises this here https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/923174 and its working on a debian-12 system proving out the pep668 stuff.

For all the openstack-ansible jobs i've now defaulted to using a venv on the kubernetes target hosts for all distributions and it looks OK.

Interested to know if you're happy with all these new vars in defaults/main.yml, or if we should hide some of them away in vars/main.yml to keep the surface area of the role defaults smaller.

[noonedeadpunk]

Can somebody re-run the job? As failure there is obviously intermittent and unrelated, as it failed to fetch available k8s versions.

Or maybe @jrosser you can rebase on top of main to trigger it?

[jrosser]

Or maybe @jrosser you can rebase on top of main to trigger it?

Done.

[noonedeadpunk]

@mnaser is there any chance to get this patch in? as in OSA we pretty much relying on venvs, so not having ability to use them is pretty much a blocker for moving on.

[mnaser]

@noonedeadpunk inthink mostly what would be nice is CI for this and we have noble nodes available for use to be able to test it..

[noonedeadpunk]

Ok, I will try to take a look at way of testing this. So far - we perform test of this branch in our ops repo:
https://zuul.opendev.org/t/openstack/build/dad032e2af4d4905ae9409db3dbe61da

https://opendev.org/openstack/openstack-ansible-ops/src/branch/master/mcapi_vexxhost/playbooks/files/openstack_deploy/user-collection-requirements.yml#L2-L5

[jrosser]

I think that the main issue is that the existing CI for this is using tox which creates it's own venv, and for that reason any pip install just works becasue this is not the system python.

[noonedeadpunk]

also, another rebase would be helpful for me at least - just yesterday I tried to run control cluster at 1.30 and made a very loud "doh" as that would need both this PR and !137

[jrosser]

@noonedeadpunk this is rebased again now

[mnaser]

So this is something that we can finally approach properly and see the errors/failures because we have actual an actual remote connection, so this means this issue actually shows up, rather than Ansible using the same as the virtual env we use to run it on the same host (since we used connection = local).

I am not a fan of this approach, due to the fact that it means that using the kubernetes.core.k8s module cannot be used easily because now you'd have to update all your references to point to that arbitrary virtual environment, which makes consuming this difficult.

The approach I'm looking into is packaging the Python SDK for Kubernetes which should resolve this issue (for now).

[mnaser]

#191 should be the best way forward that maintains everything to continue working, and we use the modules and they're clearly just working fine.

[mnaser]

solved by #191