[Docs] Clarify expected interface for running distutils.commands

[albertosottile]

Summary

I am the maintainer of py2exe and I am in the process of migrating it from distutils to setuptools. While reading the new documentation, I found out this extremely useful page that carefully explains how to add a new command and even new keywords for the setup() method, effectively mimicking what has been possible up to now with distutils. If this API is supposed to be actively supported in the future, changes on py2exe would most likely be relatively minimal.

However, at the end of the very same page, under Final Remarks there is this sentence:

Directly calling python setup.py ... is considered a deprecated practice. You should not add new commands to setuptools expecting them to be run via this interface.

which seems to contradict the point of the whole page. I think I am missing something here: what is then the expected way to run a distutils.commands entry point? Do you plan to discontinue the python setup.py ... command interface?

Thanks for this clarification and for maintaining setuptools.

OS / Environment

No response

Additional Information

The linked page should clarify what is the command line interface that should be used to run commands added to setuptools via the supported distutils.commands entry point.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[abravalheri]

Hi @albertosottile thank you very much for opening this discussion.

I think I am missing something here: what is then the expected way to run a distutils.commands entry point?

Command objects are useful in different contexts, not only when called via the CLI. Two major examples here would be when an user wants to add a custom build step (e.g. transpiling typescript files into javascript files to be embedded in a wheel for distribution) or when a plugin wants to overwrite one of the default build steps (e.g. build_py, build_ext).

Do you plan to discontinue the python setup.py ... command interface?

I believe that this is the long term plan. I joined the project as a maintainer after this decision was made, but so far there are many items in the changelog point to this direction. One of the setuptools maintainers has created this article explaining the situation.

The tendency is that setup.py will be reserved as an internal implementation detail for setuptools own use, instead of being exposed as a public interface.

The linked page should clarify what is the command line interface that should be used to run commands added to setuptools via the supported distutils.commands entry point.

As far as I am aware, the is no plan to support a command line interface to run custom commands.

[ronaldoussoren]

Is there a timeline on when the python setup.py ... interface will be removed, in particular for custom commands?

I'm the maintainer for py2app that uses a similar design as py2exe for the command line interface: Use "python setup.py py2app" to build an application based on configuration in the setup.py file.

[abravalheri]

I am really sorry @ronaldoussoren I don't have that information for you.

I joined the team after setuptools started giving indications that it wanted to go in this direction, so there are many things I don't know yet...

[albertosottile]

@abravalheri Sorry if I keep asking about this, but I am still trying to find the best/suggested approach for changing this in py2exe.

Command objects are useful in different contexts, not only when called via the CLI. Two major examples here would be when an user wants to add a custom build step (e.g. transpiling typescript files into javascript files to be embedded in a wheel for distribution)

Suppose that, in this scenario, the user wants to run just the transpiling step, without building the full wheel. How is the user supposed to do that? What would be the preferred way to create/expose a CLI for running this transpiling step (internally implemented as a setuptools command)?

[abravalheri]

Suppose that, in this scenario, the user wants to run just the transpiling step, without building the full wheel. How is the user supposed to do that? What would be the preferred way to create/expose a CLI for running this transpiling step (internally implemented as a setuptools command)?

I believe that the direction setuptools has chosen to follow (even before I joined as a maintainer) is to use standardized workflows (e.g. the one described in PEP 517) and deprecate exposing CLI capabilities. This definitely limits what can be done (not everything will work, e.g. it is a clear intention of the project that users don't run the install "build step").

Therefore, we can infer that this goal would have to be achieved via Python APIs instead of running python setup.py in a sub-processes.

The safest way of doing that is to create a third-party CLI tool that calls the build hooks in setuptools.build_meta as specified in PEP 517, if possible. This set of APIs is definitely the most stable interface setuptools offers since they are closely tied to the PyPA specs.

Projects like enscons, also use the Command(dist_obj).run API. Note however that (despite not currently planned) setuptools eventually may introduce API changes in future major versions (e.g. add a deprecation warning, changes in arguments, etc...). This is a natural part of the life cycle of software (it is impossible to foresee the future), so standard practices to protect against dependency changes are applicable depending on how conservative/reactive you want the project to be (e.g. adding strategic major version caps that are bumped after testing recently published versions).

---

The original title of this issue is [Docs] Clarify expected interface for running distutils.commands. My understanding is that setuptools effectively deprecated built-in user-facing mechanisms of doing so, therefore I am not sure if there is something to be documented here. Currently there is an entry in the "deprecated practice" docs with a warning section specifically alerting for this deprecation: https://setuptools.pypa.io/en/latest/deprecated/commands.html.

Alternatively, we could interpret the issue as a request to add documentation about lower-level setuptools Python APIs (such as the aforementioned Command(dist_obj).run). Would that be the case?
If so, this issue might be a duplicate of #3401 since it specifically mentions API Reference, and related to #2387. A dependency to proceed with that issue is to build consensus and clarify which parts of the low level API are intended and which parts of the API are accidental.

[merwok]

setuptools effectively deprecated built-in user-facing mechanisms of doing so

Could you specify who user is in this sentence?

IMO, it’s fine for the developers of the project to use commands specific to their build tools, like python setup.py build_js, even if people who want to use the project should not use setup.py directly. Otherwise there’s a lot of work needed for setuptools to offer another command API, and the benefits of that are dubious.

[abravalheri]

Hi @merwok, the best reference that we have so far is linked post. I can highlight the following parts:

all direct invocations of setup.py are effectively deprecated in favor of invocations via purpose-built and/or standards-based CLI tools like pip, build and tox.

The setuptools project no longer wants to provide any public CLI, and will be actively removing the existing interface (though the time scale for this is long).

So presumably this applies to every single invocation of python setup.py that is outside of the setuptools code base (and users refers to anyone including developers).

[albertosottile]

The original title of this issue is [Docs] Clarify expected interface for running distutils.commands. My understanding is that setuptools effectively deprecated built-in user-facing mechanisms of doing so, therefore I am not sure if there is something to be documented here.

IMHO, because the documentation invests a lot of effort in explaining how to extend setuptools commands, it should also specify what is the "supported/recommended" way in which these commands should be executed. I would be in favor of any addition to the documentation based on either this approach

The safest way of doing that is to create a third-party CLI tool that calls the build hooks in setuptools.build_meta as specified in PEP 517, if possible.

or this approach

Projects like enscons, also use the Command(dist_obj).run

or even something else entirely.

The goal would be that maintainers of other projects that currently depend on the setup.py CLI know exactly where they should go when departing from distutils.

[ronaldoussoren]

FYI: There's a blog about the deprecation here: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html