Inference of setuptools' own version from git history (without setuptools-scm) #4948

abravalheri · 2025-04-16T18:23:24Z

Summary of changes

This is a follow-up (and alternative approach to) #4537, #4532 and try to address some of the shortcomings identified.

The approach implemented is to call directly git describe --abbrev --match 'v?[0-9]*' --dirty falling back to read the latest version from NEWS.rst and adding a "dirty" local time-based version.

Potential points of discussion:

What happens if a "downstream" packager simply download a tar ball from Github or performs a shallow clone without tag history?

In this PR, the version would be determined as follows:
1. The custom function will read the latest version from NEWS.rst
2. The custom function will add a time-based development marker (similar to something that already happens prior to the PR: https://github.com/pypa/setuptools/blob/v78.1.0/setup.cfg#L1-L3)
So downstream packagers may see a "non-pristine"/"dirty" version string if they don't have access to the commit/tag history, which may be undesirable, but not prohibitive.
Are we reimplemented and "half-backing" functionality already present in setuptools-scm?

Well, yes. But we take advantage that version tags in setuptools are very predictable to simplify the code and that we already have implemented some functionality that allow us to shoehorn malformed version strings into compliant ones.

This means that we can implement the functionality with a couple of lines¹ of code assuming the installed version of git supports the command git describe --abbrev --match 'v?[0-9]*' --dirty (which unless I am wrong is available at least since version 2.0.5 of git: https://git-scm.com/docs/git-describe/2.0.5).

On top of that, this PR implements a functionality not present on setuptools-scm (AFAIK), extract a fallback version from NEWS.rst. All things considered the custom function implementation is implemented with a couple of lines¹ of code.

Closes #4530

Pull Request Checklist

Changes have tests
News fragment added in newsfragments/.
(See documentation for details)

The amount of lines necessary increased after consider fallback to PKG-INFO to ensure version consistency better sdist/wheel. ↩ ↩²

abravalheri · 2025-04-16T18:29:15Z

.github/workflows/main.yml

@@ -71,6 +71,8 @@ jobs:
    timeout-minutes: 75
    steps:
      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0


In the absence of the commit/tag history it is not possible to infer the version correctly. So we need a deep checkout.

This has been discussed in https://github.com/pypa/setuptools/pull/4537/files#r1701658826.

The approach seems to be the standard workaround for the same problem in setuptools-scm, see pypa/setuptools-scm#480, pypa/setuptools-scm#952 (comment). Tracked in actions/checkout#249 and actions/checkout#1471.

abravalheri · 2025-04-16T18:30:40Z

setup.py

+    except subprocess.CalledProcessError:  # e.g.: git not installed or history missing
+        with open("NEWS.rst", encoding="utf-8") as fp:
+            match = re.search(r"v\d+\.\d+.\d+", fp.read())  # latest version
+            return f"{match[0] if match else '0.0.0'}.dev+{time.strftime('%Y%m%d')}"


%Y%m%d can be changed to more specific time to improve monotonicity... Please let me know if a more specific tag would be better.

BTW, the except is the code path that takes effect when git tag version is not available (tarballs or shallow clones).

I have considered checking if a PKG-INFO file is available but that would increase the complexity of this function... Please let me know if this more complex approach should be followed...

OK, it may be necessary to check for PKG-INFO, because otherwise we may see inconsistency between sdist and wheel the following behaviour when running python -m build:

$ python -m build Successfully built setuptools-78.1.2.tar.gz and setuptools-78.1.2.dev0+20250416-py3-none-any.whl

OK, it may be necessary to check for PKG-INFO, because otherwise we may see the following behaviour when running python -m build:

So I inevitably had to add some lines in aee4dc9 to deal with this problem:

https://github.com/pypa/setuptools/pull/4948/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R39-R42

It does increase the complexity of the solution.

We can see now consistency between version inference in sdist and wheel: https://github.com/pypa/setuptools/actions/runs/14501313828/job/40681414420?pr=4948#step:8:3078

abravalheri · 2025-04-16T18:31:49Z

setuptools/_normalization.py

@@ -12,7 +12,7 @@
 _VALID_NAME = re.compile(r"^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$", re.I)
 _UNSAFE_NAME_CHARS = re.compile(r"[^A-Z0-9._-]+", re.I)
 _NON_ALPHANUMERIC = re.compile(r"[^A-Z0-9]+", re.I)
-_PEP440_FALLBACK = re.compile(r"^v?(?P<safe>(?:[0-9]+!)?[0-9]+(?:\.[0-9]+)*)", re.I)
+_PEP440_FALLBACK = re.compile(r"^(?P<safe>v?(?:[0-9]+!)?[0-9]+(?:\.[0-9]+)*)", re.I)


This is an existing bug that I have identified when working on this PR.
I added a new doctest to avoid regression down in the same file:

>>> best_effort_version("v78.1.0-2-g3a3144f0d") '78.1.0.dev0+sanitized.2.g3a3144f0d'

abravalheri · 2025-04-16T18:32:45Z

setuptools/_normalization.py

-def best_effort_version(version: str) -> str:
+def best_effort_version(
+    version: str,
+    template: str = "{safe}.dev0+sanitized.{sanitized}",


This customisable template parameter is not strictly necessary, it can be removed if we don't mind versions looking like 78.1.0.dev0+sanitized.2.g3a3144f0d

abravalheri · 2025-04-16T18:33:05Z

tox.ini

@@ -102,8 +101,6 @@ setenv =
 	TWINE_USERNAME = {env:TWINE_USERNAME:__token__}
 commands =
 	python -c "import shutil; shutil.rmtree('dist', ignore_errors=True)"
-	# unset tag_build and tag_date pypa/setuptools#2500
-	python setup.py egg_info -Db "" saveopts


I think we would no longer need this.

abravalheri · 2025-04-16T18:41:01Z

Possibly needs more tests, but posting to collect feedback if the direction is viable.

abravalheri · 2025-04-16T20:11:51Z

Error with pytest-mypy in 3.14 seems unrelated:

abravalheri · 2025-04-16T20:18:20Z

@jaraco would this kind of approach be something acceptable, taking into consideration the previous discussions in #4530, #4537, #4532.

It is a bit of reimplementing functionality.

… starting with v

abravalheri · 2025-05-29T10:47:04Z

setup.py

+        return _normalization.best_effort_version(version, "{safe}.dev+{sanitized}")
+    except subprocess.CalledProcessError:  # e.g.: git not installed or history missing
+        candidates = filter(None, starmap(_extract_version, _VERSION_FALLBACK))
+        return next(candidates, f"0.0.0.dev+{time.strftime('%Y%m%d')}")


Reposting from thread in https://github.com/pypa/setuptools/pull/4948/files#r2047494928:

The exact format of the version can be further discussed, e.g. %Y%m%d can be changed to more specific time to improve monotonicity... Please let me know if a more specific tag would be better.

The except is the code path that takes effect when git is not installed and/or tag is not available (tarballs or shallow clones).

Reading PKG-INFO ensures consistency between sdists/wheels:

Reading NEWS.rst is a fallback when tags are missing from shallow clones (or someone is using a git tarball without sdist metadata).

I cannot see 0.0.0 being produced unless there is something wrong with the repo (e.g. missing NEWS.rst file), but it is added to make static analysis happy.

Custom function results should be similar to setuptools-scm but without using setuptools-scm.

Should be no longer necessary now that `setup.cfg` is gone.

…rom sdist

abravalheri commented Apr 16, 2025

View reviewed changes

abravalheri changed the title ~~Auto version~~ Inference of setuptools' own version from git history Apr 16, 2025

abravalheri changed the title ~~Inference of setuptools' own version from git history~~ [WIP] Inference of setuptools' own version from git history Apr 16, 2025

abravalheri requested a review from jaraco April 16, 2025 20:18

Fix extra character in _normalization.best_effor_version for versions…

f6f2cc2

… starting with v

abravalheri force-pushed the auto-version branch from 8573e95 to 79f4c12 Compare May 29, 2025 10:40

abravalheri commented May 29, 2025

View reviewed changes

abravalheri changed the title ~~[WIP] Inference of setuptools' own version from git history~~ Inference of setuptools' own version from git history May 29, 2025

abravalheri marked this pull request as ready for review May 29, 2025 11:00

abravalheri changed the title ~~Inference of setuptools' own version from git history~~ Inference of setuptools' own version from git history (without setuptools-scm) May 29, 2025

abravalheri and others added 8 commits May 29, 2025 12:01

Allow custom template in _normalization.best_effor_version

2ff5a0e

Use dynamic version for setuptools itself

346ae7e

Custom function results should be similar to setuptools-scm but without using setuptools-scm.

Remove bumpversion and rely on jaraco.develop for finalize operation.

072df10

Remove 'setup.py saveopts' step from tox.ini

28d4522

Should be no longer necessary now that `setup.cfg` is gone.

Prevent shallow history in GHA to mess with version inference

d50b3cb

Ensure version inference falls back to PKG-INFO when building wheel f…

723ca92

…rom sdist

Add news fragment

f1956af

Refactor _get_version in setup.py

06343cf

abravalheri force-pushed the auto-version branch from 79f4c12 to 06343cf Compare May 29, 2025 11:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Inference of setuptools' own version from git history (without setuptools-scm) #4948

Inference of setuptools' own version from git history (without setuptools-scm) #4948

Uh oh!

abravalheri commented Apr 16, 2025 •

edited

Loading

Uh oh!

abravalheri Apr 16, 2025

Uh oh!

abravalheri Apr 16, 2025

Uh oh!

abravalheri Apr 16, 2025 •

edited

Loading

Uh oh!

abravalheri Apr 16, 2025 •

edited

Loading

Uh oh!

abravalheri Apr 16, 2025 •

edited

Loading

Uh oh!

abravalheri Apr 16, 2025

Uh oh!

abravalheri Apr 16, 2025

Uh oh!

abravalheri Apr 16, 2025

Uh oh!

abravalheri commented Apr 16, 2025

Uh oh!

abravalheri commented Apr 16, 2025 •

edited

Loading

Uh oh!

abravalheri commented Apr 16, 2025

Uh oh!

abravalheri May 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Inference of setuptools' own version from git history (without setuptools-scm) #4948

Are you sure you want to change the base?

Inference of setuptools' own version from git history (without setuptools-scm) #4948

Uh oh!

Conversation

abravalheri commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Pull Request Checklist

Footnotes

Uh oh!

abravalheri Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri commented Apr 16, 2025

Uh oh!

abravalheri commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abravalheri commented Apr 16, 2025

Uh oh!

abravalheri May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

abravalheri commented Apr 16, 2025 •

edited

Loading

abravalheri Apr 16, 2025 •

edited

Loading

abravalheri Apr 16, 2025 •

edited

Loading

abravalheri Apr 16, 2025 •

edited

Loading

abravalheri commented Apr 16, 2025 •

edited

Loading

abravalheri May 29, 2025 •

edited

Loading