pip freeze strips VCS credential env vars

[connesy]

Description

Installing from a VCS URL that included credentials as environment variables, those variables are stripped from the "url" in direct_url.json, meaning they are also not included in the output of pip freeze.

Original comment: #11410 (comment)

Expected behavior

I would expect the "url" field in direct_url.json to include the environment variables that are used for authentication.

pip version

22.0.2

Python version

3.10.12

OS

Ubuntu 22.04

How to Reproduce

1. Create a venv virutal environment
2. Activate the environment
3. Add a VCS URL containing environment variables to a requirements.txt file
4. Run pip install -r requirements.txt
5. Inspect direct_url.json or run pip freeze

Output

$ mkdir pip-test; cd pip-test
$ echo "resolvelib @ git+https://${TEST_USER}:${TEST_PASS}@github.com/sarugaku/resolvelib@1.0.1" > requirements.txt
$ python3 -m venv .venv
$ . .venv/bin/activate
$ export TEST_USER="test-user"
$ export TEST_PASS="test-pass"
$ pip install -r requirements.txt
Collecting resolvelib@ git+https://:@github.com/sarugaku/resolvelib@1.0.1
  Cloning https://:****@github.com/sarugaku/resolvelib (to revision 1.0.1) to /tmp/pip-install-jq6_4h_8/resolvelib_b660f6e7cd70411981cf94b5c4a10857
  Running command git clone --filter=blob:none --quiet 'https://:****@github.com/sarugaku/resolvelib' /tmp/pip-install-jq6_4h_8/resolvelib_b660f6e7cd70411981cf94b5c4a10857
  warning: redirecting to https://github.com/sarugaku/resolvelib/
  Running command git checkout -q 969c5c75ded426dac43eff9619e1e857f22f37bb
  warning: redirecting to https://github.com/sarugaku/resolvelib/
  Resolved https://:****@github.com/sarugaku/resolvelib to commit 969c5c75ded426dac43eff9619e1e857f22f37bb
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: resolvelib
  Building wheel for resolvelib (pyproject.toml) ... done
  Created wheel for resolvelib: filename=resolvelib-1.0.1-py2.py3-none-any.whl size=17195 sha256=b4352dc966ecd7ddfd72a009d6f370860bff476027ab34990265cdbf5e8aa2f7
  Stored in directory: /tmp/pip-ephem-wheel-cache-e4my4hf4/wheels/37/d4/ac/771993aa3deae60395d34f040f543177533359d0efa0034ecf
Successfully built resolvelib
Installing collected packages: resolvelib
Successfully installed resolvelib-1.0.1
$ python -m json.tool .venv/lib/python3.10/site-packages/resolvelib-1.0.1.dist-info/direct_url.json
{
    "url": "https://github.com/sarugaku/resolvelib",
    "vcs_info": {
        "commit_id": "c9ef371ad96e698bf3e0bb09acc682bd43e39bd7",
        "requested_revision": "1.0.1",
        "vcs": "git"
    }
}
$ pip freeze
resolvelib @ git+https://github.com/sarugaku/resolvelib@c9ef371ad96e698bf3e0bb09acc682bd43e39bd7

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

I believe this is intended since the auth information is considered sensitive.

[connesy]

I am aware that plaintext username and passwords are considered sensitive, but according to the PEP 610 spec, auth information given as environment variables may be included in direct_url.json:

The user:password section of the URL MAY however be composed of environment variables [...]

https://peps.python.org/pep-0610/#:~:text=for%20security%20reasons.-,The%20user%3Apassword%20section%20of%20the%20URL,-MAY%20however%20be

Also, according to #11410 (comment) pip should already do this.

[uranusjr]

May means truly optional so it is up to pip to decide whether to implement it. It does not even imply a recommendation, so at least the bug category is wrong. I’ll defer to @sbidoul on whether it is a good idea for pip to implement this.

[connesy]

Sorry, I filed this as a bug report as I got the impression from @sbidoul that this should already be a feature of pip, as discussed in the original issue thread here: #11410 (comment).

That aside, do you see any security issues with implementing this, as it would only be the names of the environment variables that would be included in direct_url.json?
As you pointed out yourself here, pip freeze is supposed to reflect the installation, and in this case the auth part is lost even if it is not (IMO) sensitive data.

[sbidoul]

I have reclassified this as a feature request.
It was indeed my intent when writing PEP 610 and I honestly thought it did work.
The solution is non trivial, though, so don't hold your breath for it.

Out of curiosity, would a .netrc file would work for your use case?

[connesy]

@sbidoul Thanks for the suggestion - I think I can make it work with .netrc.

However, I would still love to see this as part of pip itself, exactly because of the expectation that pip freeze reflects the installation.