Consider accepting `-` and `_` in extras name

[jamesliu4c]

Here is the changelog message for 78.0.2:

Postponed removals of deprecated dash-separated and uppercase fields in setup.cfg. All packages with deprecated configurations are advised to move before 2026.

I advocate that the better option is to accept - and _, and remove the deprecation warning. Anything else would risk major breaking changes throughout the ecosystem again.

I traced down the issue where making the deprecation into a breaking change was discussed. #4864 (comment).

Quoting rather liberally,

Now we have a couple of options:

1. Transform the deprecation warning in a proper error, to force the end-users to implement the necessary changes in their code-base.
2. Postpone it if we feel that the community is not ready for the change yet.
3. Give-up entirely on the deprecation on the basis that there might be unmaintained packages out there only published to PyPI as sdists important to the environment.
   a. We can just settle for forever warning the deprecation without removing backwards compatibility.
   b. Change the parsing of setup.cfg accept both - or _ without warnings.

And @jaraco said in response:

That said, I'm leaning toward option 1, get some signal out there to non-compliant projects, and then if the disruption is large, rollback and fallback to option 2 (now with the affected projects having visibility to the issue).

The ticket was closed when the tests were marked as xfail here.

@zanieb warned that the disruption would be quite large here:

That said, I'm leaning toward option 1, get some signal out there to non-compliant projects,

I think rolling out changes like this is quite disruptive to the ecosystem and is to aggressive for a foundational package like this. I would appreciate more effort determining if changes like this would be disruptive ahead of time.
...

Change the parsing of setup.cfg accept both - or _ without warnings.

Is there a downside to this approach? It looks like this was originally deprecated because the extra names were not installed correctly by pip (#1608) but that doesn't seem like a strong motivation? (I tried pip install '.[foo-b]' with the extra name foo_b and it seems to work fine now)

And now, after all this, we are at option 2, punt. Sometime in 2026, we will do this all over again. Don't let's kid ourselves that the long tail of PyPI packages will release patches to comply with this change within a year.

So why not choose option 3, remove the deprecation, and roll forward accepting - and _?

[abravalheri]

By all means, please do propose a PR (or series of PRs). The postponed error gives us the opportunity to rethink the approach.

I think this is a good opportunity to delve into the real problems here and try to come up with good solutions.

One think to consider before attempting to tackle the challenge on the title of the issue is that we learned with the community reports that the vast majority of errors are happening specifically for fields that are not even accepted by setuptools. This helped to surface a hidden issue #4913 and I think this is something more problematic that we need to solve.

This is what I consider important when tackling this request and the preferred direction for the PR:

• The handling should go to the setuptools.config.setupcfg file. Rationale: right now, we have code caring for the parsing scattered into a couple of modules. This makes maintenance hard and introduce shotgun surgery code smell.
• Simplify the complexity of the existing code base.
• We should consider the impact of this change in the handling of [BUG] Setuptools allow arbitrarily wrong configs in setup.cfg #4913. We should not accept invalid fields regardless if they use dashes or underscores (of course the way this should be handled is gradual)
• Asses the impact of the proposed change in the context of why the deprecation was introduced a couple of years ago.

Finally, I would assume the person proposing the PR is willing to support the implementation at least during the period of time that it will take for it to reach stable levels.

---

Note that the comment @zanieb warned that the disruption would be quite large is misleading as it gives the impression that the warning was posted before the merge/tag, but it did not.

---

In hindsight, based on the feedback received, I found that in general, there is almost no usage of dashes, except for fields that are not supported by setuptools.

---

Notes not specific to this request, but an important general reminder:

Expecting that setuptools will not introduce deprecation changes in general and will not carry out the removals is unrealistic. We can try to improve communication, we can contribute togheter to write guides of what to do, and how projects can take measures to protect mission critic systems. We are also very happy to review approaches and discuss better solutions.

We have been working very hard to implement warnings as visible and as informative as we can. We implemented suggestions from the community of how deprecation warnings should be displayed. We also have been criticised for not having clarity on the removal dates planned, and we changed those. Please note that we have been taking input on board, but it is not fair to expect that things will never change.

---

Update: see #4923 (review).

[zahlman]

I agree with this suggestion.

The proposed change seems to be just a partial reversion of #2588, so it shouldn't be difficult to put together.

It appears that "not accepting invalid fields", altogether, would be far more disruptive than this normalization issue was. Something should presumably be done about that eventually, but it doesn't seem to be as high of a priority. (Have there been complaints about packages building incorrectly due to Setuptools ignoring a mis-typed key in setup.cfg?) I'd like to hope that new projects are using pyproject.toml for as much as possible and that relatively few of them have any use for setup.cfg in the first place.

Regarding the visibility of warnings, I hope @notatallshaw 's discussion will bring much-needed light. (Unfortunately I will not be able to participate in that discussion.)

[jamesliu4c]

By all means, please do propose a PR (or series of PRs). The postponed error gives us the opportunity to rethink the approach.

Ok, will do.

Note that the comment @zanieb warned that the disruption would be quite large is misleading as it gives the impression that the warning was posted before the merge/tag, but it did not.

Understood.

[jamesliu4c]

• We should consider the impact of this change in the handling of [BUG] Setuptools allow arbitrarily wrong configs in setup.cfg #4913. We should not accept invalid fields regardless if they use dashes or underscores (of course the way this should be handled is gradual)

Completely disagree with this point, though. Instead, I believe this comment from @herebebeasties is correct:

Given all the existing packages out there that have config keys that setuptools doesn't recognise, accepting those unknown keys isn't a bug, it is very much a feature.

At the very most you should log a warning that setuptools is ignoring them.

Likewise, this comment from @eli-schwartz.

setuptools should not attempt to validate fields that don't belong to it. Let them be as right or as wrong as they please, it's not setuptools' place to tell them they are doing something wrong.

Very well, projects may pass wrong config according to setuptools. Do these projects have other tools that use those configs? Maybe. Are they just making a mistake? Maybe.

I believe this is the crux of the governance changes we are seeking. Should setuptools break builds that it deems incorrect? According to #4913, the answer appears to be yes. Some vocal parts of the community (me included) believe it should be no.

Rather, I fervently wish that setuptools would hold with the robustness principle. As Hyrum's law states, "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody." Setuptools very much has a sufficient number of users.

---

[Edited to add the following]

Anyway, this is a long way of saying that I don't intend to handle 4913 with any PR.

As for this,

Expecting that setuptools will not introduce deprecation changes in general and will not carry out the removals is unrealistic. We can try to improve communication, we can contribute togheter to write guides of what to do, and how projects can take measures to protect mission critic systems. We are also very happy to review approaches and discuss better solutions.

Of course, there will have to be deprecations and removals. But we would like greater input into whether any given deprecation is really necessary. But I think that longer discussion should be in its own issue about governance. I can open that at a later time, or someone else can.

[webknjaz]

I advocate that the better option is to accept - and _, and remove the deprecation warning. Anything else would risk major breaking changes throughout the ecosystem again.

FTR, underscores are invalid in the core packaging metadata: https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use.

[jamesliu4c]

I advocate that the better option is to accept - and _, and remove the deprecation warning. Anything else would risk major breaking changes throughout the ecosystem again.

FTR, underscores are invalid in the core packaging metadata: https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use.

Very well. In that case, a better place to validate may be uploads to PyPI. Suppose nonconforming fields are rejected from upload. New packages and new versions must then conform. Meanwhile, builds continue to work. Maybe eventually, setuptools can also reject nonconforming _.

[zahlman]

I believe this is the crux of the governance changes we are seeking. Should setuptools break builds that it deems incorrect? According to #4913, the answer appears to be yes. Some vocal parts of the community (me included) believe it should be no.

My personal feeling is that this should be nuanced. Ideally builds should not be deliberately broken on simple aesthetic grounds, the way that this change came across as. Neither adding nor removing a normalization rule here seems like hardly any maintenance burden.

But on the other hand, there are conceivable removals that would carry huge benefits: in terms of clearer separation of responsibilities, performance, wheel size etc. I'm thinking in particular here of #4519, where the removal was a step towards making Setuptools function properly as purely a build backend (as opposed to a deprecated early-2010s attempt at a workflow tool that exposes a build API, which was itself originally an interface to wheel). I dream of a far future where Setuptools can shed layers of architecture because calling setup.py directly (or the functionality that offers outside of wheel-building) no longer has to be supported at all.

Of course, there will have to be deprecations and removals. But we would like greater input into whether any given deprecation is really necessary.

Unclear: who is "we"?

I start to think that in the long term, the only viable solution is a separate "LTS" branch that's bugfixes-only in perpetuity (and which becomes the assumed default for legacy projects) while "modern Setuptools" becomes able to do what it needs to. (Perhaps separate LTS versions for projects not declaring a [build-system], versus those that lack a pyproject.toml altogether. Since my interpretation of the former group is that they opted in to PEP 621 features, but are only using Setuptools out of inertia and aren't committed to its future. Whereas the latter group is fully invested in things continuing to work like they always have, as much as possible.)

[eli-schwartz]

(Perhaps separate LTS versions for projects not declaring a [build-system], versus those that lack a pyproject.toml altogether. Since my interpretation of the former group is that they opted in to PEP 621 features, but are only using Setuptools out of inertia and aren't committed to its future. Whereas the latter group is fully invested in things continuing to work like they always have, as much as possible.)

No, this is entirely backwards.

Having a [build-system] implies nothing other than opting into pep 517 / 518. Having a [project] is explicitly opting into PEP 621, since it literally is PEP 621.

As I said in #4915 (comment):

v68.1.0 is the first version of setuptools that had non-experimental support for pyproject.toml as a replacement for setup.cfg

This is a considerably more pessimized timeframe than your reference to 40.8.0

There's a big support gap between:

• 2017: pyproject.toml standardized, "modern" setuptools.build_meta backend General Availability
• 2022: setuptools implements support for PEP 621 [project]

If you want to change which setuptools versions to offer to people depending on whether they have bought into "committing to its future", you should look at people who did the latter, not the former.

Lots of people were told that we "had to" add a pyproject.toml and that it does not affect the existing workflow, but only allows pip to be more explicit about what it needs. For example, I too, have a pyproject.toml which happily coexists with setup.cfg, and the latter continues to drive all configuration in 2025 and onward. (Sorry to be the bearer of bad news, but toml sucks and setup.cfg remains the superior configuration syntax and format, and works great. For people who were already using toml, it's not much churn to twiddle some headers so that existing keys end up in [project], but moving from setup.cfg to pyproject.toml is pointless churn and a readability regression. Yes, our CI even tests the python setup.py install codepath, and we do accept that it's our responsibility to fix our CI if, not when, it ever breaks.)

[abravalheri]

In #4923 (review) I listed some of my thoughts related to the fact that while I would consider accepting -, I don't think that the solution is simply remove the deprecation warning or replace it with something purely informational.

[zahlman]

I don't think that the solution is simply remove the deprecation warning or replace it with something purely informational.

... Are there actually any in-between alternatives?

Or else, what is the specific concern in #4913 , anyway? The best interpretation I can make is that you're concerned that someone might write (or have written 10+ years ago...) something intended for another tool and not for the setup call, but which does (when normalized) match a setup argument name... ? But surely in this case, there would have already been complaints about the existing behaviour... ?

[webknjaz]

... Are there actually any in-between alternatives?

When talking about extras specifically (I have a feeling that some comments refer to other configurations of setuptools too, but I'm not talking about that), in order to produce spec-compliant metadata, it shouldn't include underscores in said metadata. But the configuration file might allow those names, while converting them into spec-compliant format. It's probably a good idea to just normalize extra names in the configs, as long as the metadata is correct.