Replacing optional-dependencies with dependency-groups

[egarciamendez]

Summary

• Migrated from optional-dependencies to modern PEP 735 dependency groups in pyproject.toml
• Added convenience all group that includes both dev and docs dependencies using include-group syntax
• Removed obsolete requirements-docs.txt file as dependencies are now managed through dependency groups

Changes

• Replaced [project.optional-dependencies] with [dependency-groups] section
• Created dev group with pinned development dependencies (coverage, pytest, pre-commit, etc.)
• Created docs group with documentation dependencies (sphinx, furo theme, etc.)
• Added all group using {include-group = "dev"} and {include-group = "docs"} syntax
• Removed requirements-docs.txt as theme is now included in docs dependency group

Benefits

• Modernizes dependency management using latest Python packaging standards
• Provides cleaner separation between different types of dependencies
• Enables installing all development dependencies with single command: pip install -e .[all]
• Maintains backward compatibility with existing .[dev] and .[docs] installation patterns

Test plan

• Verify pip install -e .[dev] installs development dependencies
• Verify pip install -e .[docs] installs documentation dependencies
• Verify pip install -e .[all] installs both dev and docs dependencies
• Test documentation build process works with new dependency groups
• Confirm CI/CD processes work with updated dependency structure

Fixes #2387

PR Checklist:

• All new features have been tested
• All new features have been documented
• I have read the CONTRIBUTING.md file
• I will abide by the code of conduct

[egarciamendez]

The contribution documentation needs to be changed to describe how to install the dependency groups

[egarciamendez]

This PR cannot be merged immediately due to a dependency group compatibility issue with the remote CI pipelines.

Problem

The changes in this PR migrate from optional-dependencies to modern PEP 735 dependency-groups in pyproject.toml. However, the current remote CI infrastructure does not support the newer dependency group syntax, which cause pipeline failures.

Migration Strategy

To safely implement this change, we need a 5-step approach:

Step 0: ✅ (Current PR)

• Update pyproject.toml to use [dependency-groups] instead of [project.optional-dependencies]
• Update tox.ini to use dependency_groups = dev|docs instead of extras = dev|docs
• Test locally to ensure dependency groups work correctly
• Verify all development workflows function with new dependency structure

Step 1: Change to local CI pipelines where dependency-groups are used instead to install all dependencies

Step 2: Merge This PR ⏳

Step 3: Change back to remote CI pipelines.

Step 4: Change documentation to reflect the changes in the way dependencies are being installed.

[freakboy3742]

Another pre-requisite that occurred to me: We need dependabot to support dependency groups - otherwise our development dependencies won't get bumped as the ecosystem evolves. See dependabot/dependabot-core#10847 for the tracking issue.

[Rjvs]

Just noting that dependabot support exists as of dependabot/dependabot-core#12720

[freakboy3742]

I'm going to close this PR, as it has effectively been superseded by #2502; thanks for your work on this issue.