__package_pip: explorer/state breaks when --requirement is not package name

[4nd3r]

ander@t490s:~$ pip list | grep mkosi
mkosi               12
ander@t490s:~$ echo __package_pip mkosi --requirement git+https://github.com/systemd/mkosi.git | cdist config -i - localhost
INFO: [30773]: localhost: Starting configuration run
INFO: [30773]: localhost: Processing __package_pip/mkosi
INFO: [30773]: localhost: Finished successful run in 6.26 seconds
ander@t490s:~$ cdist-dump localhost | grep mkosi
./messages:__package_pip/mkosi:installed
./object/__package_pip/mkosi/.cdist-akj58qn2/parameter/state:present
./object/__package_pip/mkosi/.cdist-akj58qn2/parameter/requirement:git+https://github.com/systemd/mkosi.git
./object/__package_pip/mkosi/.cdist-akj58qn2/explorer/state:absent
./object/__package_pip/mkosi/.cdist-akj58qn2/explorer/requirement:git+https://github.com/systemd/mkosi.git
./object/__package_pip/mkosi/.cdist-akj58qn2/explorer/pip:/usr/bin/pip3
./object/__package_pip/mkosi/.cdist-akj58qn2/code-remote:/usr/bin/pip3 install -q 'git+https://github.com/systemd/mkosi.git'
./object/__package_pip/mkosi/.cdist-akj58qn2/stderr/code-remote:WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

probably because of this:

pip_python -c '__import__("pkg_resources").require(__import__("sys").argv[1])' \
    "${requirement}" >/dev/null 2>&1 \
&& echo present \
|| echo absent

somehow it went unnoticed in #37 review 😟

[sideeffect42]

Hmm, it should work when the requirement contains the package name (__package_pip mkosi --requirement 'mkosi@git+https://github.com/systemd/mkosi.git').

"Installed" detection is a mess when using URLs.
pip install will also happily reinstall every time when given a URL while it will say "Requirement already satisfied" when given a name.

The pkg_resources check works with mkosi@ but it won't catch if e.g. the @branch has changed in the URL.

Update: I did some more testing and we should probably not feed URLs to pkg_resources.require().
So my suggestion would be to try to detect URLs (how?) and if a URL was given, use the __object_id instead.
Switching branches or doing upgrades by changing Git tags or something won't work this way, though.

Does pip even track the installation URL?

[4nd3r]

The pkg_resources check works with mkosi@ but it won't catch if e.g. the @Branch has changed in the URL.

If this technical limitation is documented, then it's fine.

So my suggestion would be to try to detect URLs

Only support __package_pip mkosi --requirement 'mkosi@git+https://github.com/systemd/mkosi.git' format and split it by @ with caveat that this way only first time install is possible.

Or does pip support something like foo[bar,baz]@git+https://github.com/foo/bar?

[sideeffect42]

Only support __package_pip mkosi --requirement 'mkosi@git+https://github.com/systemd/mkosi.git' format and split it by @ with caveat that this way only first time install is possible.

I did some research and I'm not so sure about the URL parsing anymore.

The PEP508-based pkg@url syntax is only supported by Pip >= 10.0.0b1 (2018-03-31).
e.g. Ubuntu 18.04 ships Pip 9.0.1 as python3-pip, so it wouldn't work there.

Or does pip support something like foo[bar,baz]@git+https://github.com/foo/bar?

Yes, it does.