Skip to content

[WIP] Inference of setuptools' own version from git history #4948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .bumpversion.cfg

This file was deleted.

8 changes: 8 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ jobs:
timeout-minutes: 75
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the absence of the commit/tag history it is not possible to infer the version correctly. So we need a deep checkout.

This has been discussed in https://github.com/pypa/setuptools/pull/4537/files#r1701658826.

The approach seems to be the standard workaround for the same problem in setuptools-scm, see pypa/setuptools-scm#480, pypa/setuptools-scm#952 (comment). Tracked in actions/checkout#249 and actions/checkout#1471.

- name: Install build dependencies
# Install dependencies for building packages on pre-release Pythons
# jaraco/skeleton#161
Expand Down Expand Up @@ -183,6 +185,8 @@ jobs:
timeout-minutes: 75
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Cygwin with Python
uses: cygwin/cygwin-install-action@v4
with:
Expand Down Expand Up @@ -244,6 +248,8 @@ jobs:
timeout-minutes: 75
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install OS-level dependencies
run: |
sudo apt-get update
Expand All @@ -269,6 +275,8 @@ jobs:
timeout-minutes: 75
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Python
uses: actions/setup-python@v5
with:
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ include msvc-build-launcher.cmd
include mypy.ini
include pytest.ini
include tox.ini
include NEWS.rst
include setuptools/tests/config/setupcfg_examples.txt
include setuptools/config/*.schema.json
global-exclude *.py[cod] __pycache__
4 changes: 4 additions & 0 deletions newsfragments/4948.misc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Setuptools stopped using ``egg_info --tag-build --tag-date`` for its own build
process. Instead version is now inferred from git tag history.
To ensure the proper version is considered, please make sure to fetch the git tags.
Local development tags are added to the latest version if the latest commit is not tagged.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ backend-path = ["."]

[project]
name = "setuptools"
version = "78.1.0"
authors = [
{ name = "Python Packaging Authority", email = "[email protected]" },
]
Expand All @@ -26,6 +25,7 @@ requires-python = ">=3.9"
dependencies = [
]
keywords = ["CPAN PyPI distutils eggs package management"]
dynamic = ["version"]

[project.urls]
Source = "https://github.com/pypa/setuptools"
Expand Down
3 changes: 0 additions & 3 deletions setup.cfg

This file was deleted.

20 changes: 20 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
#!/usr/bin/env python

import os
import re
import subprocess
import sys
import textwrap
import time

import setuptools
from setuptools import _normalization
from setuptools.command.install import install

here = os.path.dirname(__file__)
Expand All @@ -26,6 +30,21 @@
package_data.setdefault('setuptools.command', []).extend(['*.xml'])


def _get_version() -> str:
cmd = ["git", "describe", "--abbrev", "--match", "v?[0-9]*", "--dirty"]
try:
version = subprocess.check_output(cmd, encoding="utf-8")
return _normalization.best_effort_version(version, "{safe}.dev+{sanitized}")
except subprocess.CalledProcessError: # e.g.: git not installed or history missing
if os.path.exists("PKG-INFO"): # building wheel from sdist
with open("PKG-INFO", encoding="utf-8") as fp:
if match := re.search(r"^Version: (\d+\.\d+\.\d+.*)$", fp.read(), re.M):
return match[1]
with open("NEWS.rst", encoding="utf-8") as fp:
match = re.search(r"v\d+\.\d+\.\d+", fp.read()) # latest version
return f"{match[0] if match else '0.0.0'}.dev+{time.strftime('%Y%m%d')}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%Y%m%d can be changed to more specific time to improve monotonicity... Please let me know if a more specific tag would be better.

Copy link
Contributor Author

@abravalheri abravalheri Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, the except is the code path that takes effect when git tag version is not available (tarballs or shallow clones).

I have considered checking if a PKG-INFO file is available but that would increase the complexity of this function... Please let me know if this more complex approach should be followed...

Copy link
Contributor Author

@abravalheri abravalheri Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it may be necessary to check for PKG-INFO, because otherwise we may see inconsistency between sdist and wheel the following behaviour when running python -m build:

$ python -m build
Successfully built setuptools-78.1.2.tar.gz and setuptools-78.1.2.dev0+20250416-py3-none-any.whl

Copy link
Contributor Author

@abravalheri abravalheri Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it may be necessary to check for PKG-INFO, because otherwise we may see the following behaviour when running python -m build:

So I inevitably had to add some lines in aee4dc9 to deal with this problem:

https://github.com/pypa/setuptools/pull/4948/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R39-R42

It does increase the complexity of the solution.

We can see now consistency between version inference in sdist and wheel: https://github.com/pypa/setuptools/actions/runs/14501313828/job/40681414420?pr=4948#step:8:3078



def pypi_link(pkg_filename):
"""
Given the filename, including md5 fragment, construct the
Expand Down Expand Up @@ -76,13 +95,14 @@
"""
Undo secondary effect of `extra_path` adding to `install_lib`
"""
suffix = os.path.relpath(self.install_lib, self.install_libbase)

Check warning on line 98 in setup.py

View workflow job for this annotation

GitHub Actions / pyright (3.9, ubuntu-latest)

No overloads for "relpath" match the provided arguments (reportCallIssue)

Check warning on line 98 in setup.py

View workflow job for this annotation

GitHub Actions / pyright (3.9, ubuntu-latest)

Argument of type "str | None" cannot be assigned to parameter "path" of type "StrPath" in function "relpath"   Type "str | None" is not assignable to type "StrPath"     Type "None" is not assignable to type "StrPath"       "None" is not assignable to "str"       "None" is incompatible with protocol "PathLike[str]"         "__fspath__" is not present (reportArgumentType)

Check warning on line 98 in setup.py

View workflow job for this annotation

GitHub Actions / pyright (3.13, ubuntu-latest)

No overloads for "relpath" match the provided arguments (reportCallIssue)

Check warning on line 98 in setup.py

View workflow job for this annotation

GitHub Actions / pyright (3.13, ubuntu-latest)

Argument of type "str | None" cannot be assigned to parameter "path" of type "StrPath" in function "relpath"   Type "str | None" is not assignable to type "StrPath"     Type "None" is not assignable to type "StrPath"       "None" is not assignable to "str"       "None" is incompatible with protocol "PathLike[str]"         "__fspath__" is not present (reportArgumentType)

if suffix.strip() == self._pth_contents.strip():
self.install_lib = self.install_libbase


setup_params = dict(
version=_get_version(),
cmdclass={'install': install_with_pth},
package_data=package_data,
)
Expand Down
13 changes: 9 additions & 4 deletions setuptools/_normalization.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
_VALID_NAME = re.compile(r"^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$", re.I)
_UNSAFE_NAME_CHARS = re.compile(r"[^A-Z0-9._-]+", re.I)
_NON_ALPHANUMERIC = re.compile(r"[^A-Z0-9]+", re.I)
_PEP440_FALLBACK = re.compile(r"^v?(?P<safe>(?:[0-9]+!)?[0-9]+(?:\.[0-9]+)*)", re.I)
_PEP440_FALLBACK = re.compile(r"^(?P<safe>v?(?:[0-9]+!)?[0-9]+(?:\.[0-9]+)*)", re.I)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an existing bug that I have identified when working on this PR.
I added a new doctest to avoid regression down in the same file:

    >>> best_effort_version("v78.1.0-2-g3a3144f0d")
    '78.1.0.dev0+sanitized.2.g3a3144f0d'



def safe_identifier(name: str) -> str:
Expand Down Expand Up @@ -65,7 +65,10 @@ def safe_version(version: str) -> str:
return str(packaging.version.Version(attempt))


def best_effort_version(version: str) -> str:
def best_effort_version(
version: str,
template: str = "{safe}.dev0+sanitized.{sanitized}",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This customisable template parameter is not strictly necessary, it can be removed if we don't mind versions looking like 78.1.0.dev0+sanitized.2.g3a3144f0d

) -> str:
"""Convert an arbitrary string into a version-like string.
Fallback when ``safe_version`` is not safe enough.
>>> best_effort_version("v0.2 beta")
Expand All @@ -80,6 +83,8 @@ def best_effort_version(version: str) -> str:
'0.dev0+sanitized'
>>> best_effort_version("42.+?1")
'42.dev0+sanitized.1'
>>> best_effort_version("v78.1.0-2-g3a3144f0d")
'78.1.0.dev0+sanitized.2.g3a3144f0d'
"""
# See pkg_resources._forgiving_version
try:
Expand All @@ -94,8 +99,8 @@ def best_effort_version(version: str) -> str:
safe = "0"
rest = version
safe_rest = _NON_ALPHANUMERIC.sub(".", rest).strip(".")
local = f"sanitized.{safe_rest}".strip(".")
return safe_version(f"{safe}.dev0+{local}")
fallback = template.format(safe=safe, sanitized=safe_rest).strip(".")
return safe_version(fallback)


def safe_extra(extra: str) -> str:
Expand Down
9 changes: 9 additions & 0 deletions setuptools/tests/test_setuptools.py
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,15 @@ def test_findall_missing_symlink(tmpdir):
assert found == []


def test_setuptools_own_dists_infers_version(setuptools_sdist, setuptools_wheel):
# Sanity check
# Validates that `setup.py:_get_version` works as expected
assert "0.0.0" not in setuptools_sdist.name
# Validates that `setup.py:_get_version` guarantees the fallback
# code path is finding something in PKG-INFO:
assert "0.0.0" not in setuptools_wheel.name


@pytest.mark.xfail(reason="unable to exclude tests; #4475 #3260")
def test_its_own_wheel_does_not_contain_tests(setuptools_wheel):
with ZipFile(setuptools_wheel) as zipfile:
Expand Down
62 changes: 0 additions & 62 deletions tools/finalize.py

This file was deleted.

5 changes: 1 addition & 4 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,10 @@ description = assemble changelog and tag a release
skip_install = True
deps =
towncrier
bump2version
jaraco.develop >= 7.23
pass_env = *
commands =
python tools/finalize.py
python -m jaraco.develop.finalize

[testenv:vendor]
skip_install = True
Expand Down Expand Up @@ -102,8 +101,6 @@ setenv =
TWINE_USERNAME = {env:TWINE_USERNAME:__token__}
commands =
python -c "import shutil; shutil.rmtree('dist', ignore_errors=True)"
# unset tag_build and tag_date pypa/setuptools#2500
python setup.py egg_info -Db "" saveopts
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would no longer need this.

python -m build
python -m twine upload dist/*
python -m jaraco.develop.create-github-release
Loading