pip: binary filter ignores environment markers, Hermeto could fails on excluded-platform packages

[ide-developer]

Description

When using the binary filter with arch: ppc64le (or s390x), hermeto attempts to download packages that have environment markers explicitly excluding those platforms (e.g. platform_machine != "ppc64le"). Hermeto should skip these requirements entirely based on the markers, but instead tries to find arch-specific wheels and fails with PackageRejected.

Edit : Hermeto should skip these requirements entirely based on the markers.
Hermeto falls back to fetching an sdist if available, skipping the missing wheel. However, Hermeto could potentially fail if the requirements.txt file doesn't record the hash for the respective sdist (via manual requirements.txt edits) and only records wheel hashes.

Reproducer (synthetic)

mkdir /tmp/hermeto-repro && cd /tmp/hermeto-repro

#   NOTE THE EXPLICIT HASH SPECIFIER IT WILL NOT MATCH AN SDIST
cat > requirements.txt << 'EOF'
bcrypt==5.0.0 ; implementation_name == "cpython" and platform_machine != "ppc64le" and platform_machine != "s390x" and sys_platform == "linux" \
    --hash=sha256:81e5dab254e44e82bee49bca4e593b21b7e70a48bcf15725d6588df387098e54
EOF

cat > pyproject.toml << 'EOF'
[project]
name = "repro"
version = "0.1.0"
EOF

mkdir -p /tmp/hermeto-out
podman run --rm \
  -v "$(pwd):/source:z" \
  -v /tmp/hermeto-out:/output:z \
  ghcr.io/hermetoproject/hermeto:0.51.0 \
  fetch-deps --source /source --output /output \
  '[{"type":"pip","path":".","requirements_files":["requirements.txt"],"binary":{"arch":"ppc64le","os":"linux"}}]'

Expected behavior

Hermeto should recognize that bcrypt==5.0.0 has platform_machine != "ppc64le" in its environment marker and skip it entirely when the binary filter specifies arch: ppc64le.

Actual behavior

INFO -- Processing requirement line 'bcrypt==5.0.0 ; implementation_name == "cpython" and platform_machine != "ppc64le" ...'
INFO Filtering out bcrypt-5.0.0.tar.gz due to checksum mismatch
ERROR PackageRejected: No distributions found for package bcrypt==5.0.0

Hermeto processes the requirement, tries to find a ppc64le wheel, can't find one, and fails.

[eskultety]

The report doesn't include debug logs to render the full picture, i.e. we added a fallback to sdist to treat cases like ^this one not that long ago. The report is a typical "agentic" one, IOW the reproducer is synthetic rather than something a real project would likely follow. The reason for that is

cat > requirements.txt << 'EOF'
bcrypt==5.0.0 ; implementation_name == "cpython" and platform_machine != "ppc64le" and platform_machine != "s390x" >and sys_platform == "linux" \
   --hash=sha256:81e5dab254e44e82bee49bca4e593b21b7e70a48bcf15725d6588df387098e54

... it records a single hash explicitly, i.e. whatever ^this hash maps to, it doesn't map to an sdist artifact nor does it record a hash for an sdist artifact.
Our docs is quite clear on locking deps recommendation: https://hermetoproject.github.io/hermeto/latest/pip/#requirementstxt

A good way to generate requirements.txt is via pip-compile (or uv pip compile).

pip-compile would record ALL hashes, not just for wheels, and it would record all hashes even when invoked as

 uv pip compile --generate-hashes --only-binary ":all:" --output-file=requirements.txt --no-strip-markers pyproject.toml

because --only-binary has no effect on recording hashes. For a fully resolved requirements file locked with pip-compile like ^this the fetch succeeds:

2026-05-20 11:23:30,881 INFO -- Processing requirement line 'bcrypt==5.0.0 ; implementation_name == 'cpython' and platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux''
2026-05-20 11:23:31,024 DEBUG bcrypt-5.0.0.tar.gz: using intersection of requirements-file and PyPI-reported checksums
2026-05-20 11:23:31,025 INFO Downloading 1 PyPI artifacts
2026-05-20 11:23:31,026 DEBUG aiohttp.ClientSession.get(url: https://files.pythonhosted.org/packages/d4/36/3329e2518d70ad8e2e5817d5a4cac6bba05a47767ec416c7d020a965f408/bcrypt-5.0.0.tar.gz, timeout: ClientTimeout(total=None, connect=30, sock_read=300, sock_connect=None, ceil_threshold=5), raise_for_status: True)
2026-05-20 11:23:31,076 DEBUG Download completed - https://files.pythonhosted.org/packages/d4/36/3329e2518d70ad8e2e5817d5a4cac6bba05a47767ec416c7d020a965f408/bcrypt-5.0.0.tar.gz
2026-05-20 11:23:31,079 INFO Verifying checksums of bcrypt-5.0.0.tar.gz
2026-05-20 11:23:31,079 DEBUG bcrypt-5.0.0.tar.gz: sha256 checksum matches: f748f7c2d6fd375cc93d3fba7ef4a9e3a092421b8dbf34d8d4dc06be9492dfdd
2026-05-20 11:23:31,082 DEBUG Successfully processed 'bcrypt==5.0.0 ; implementation_name == 'cpython' and platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux'' in path 'deps/pip/bcrypt-5.0.0.tar.gz'

FWIW regarding bug triage, the gap between binary filters and environment markers is real, but way the issue and the reproducer are written is something I dismiss and I'll reword it accordingly. Assigning the bug label.