pyproject: Switch to using dependency groups

[akkornel]

Within pyproject.toml, we use the project.optional-dependencies group to specify Python packages that we only need for testing, or for building documentation. Although it is OK to do by convention, it is no longer the technically-correct way to encode this information.

In October 2024, PEP 735 created Dependency Groups. The idea behind Dependency Groups is that Optional Dependencies are meant for enabling certain optional client-facing functionality. For example, if you write a text-parsing package, you might want to offer a pure-Python version and a Cython version; folks would specify the Cython version via something like pip install text-parser[cython]: In this example, cython is an optional dependency.

Most packages use optional dependencies for docs and test, even though this is not client-facing. Dependency groups provides a way for packages to specify dependencies that should not make it into distributed package metadata.

We do not have any properly-optional dependencies. So, this project should be switched over to using Dependency Groups

[akkornel]

Current status of major Python package managers' support for Dependency Groups:

• For pip, support was merged in Implement a --group option for installing from [dependency-groups] found in pyproject.toml files pypa/pip#13065, and will be released in version 25.1, and will be pip install --group XXX sudns01.
• For poetry, this is still under development in feat(cli): Support PEP 735 (Dependency Groups) python-poetry/poetry#10130.
• For uv, support is in Allow [dependency-groups] in non-[project] projects astral-sh/uv#8574, released in uv 0.4.27: uv add --group XXX sudns01.
• uv's pip-compatibility added support in add pip-compatible --group flag to uv pip install and uv pip compile astral-sh/uv#11686, released in uv 0.6.7: uv pip install --group XXX sudns01.

Once Pip 25.1 is released, we can do the switch. Both pyproject.toml and the docs will need to update.

As for packaging, this only affects dependencies for testing & building docs, not user-facing dependencies. So, as long as build-time dependencies are specified in packaging files, I think things should be fine.