Skip to content

Hatch environments support PEP735 dependency-groups #1922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/config/environment/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,71 @@ you could then run your tests consecutively in all 4 environments with:
hatch run test:cov
```

## Dependency Groups

Environments can use [PEP 735](https://peps.python.org/pep-0735/) dependency groups with the `dependency-groups` option:

```toml config-example
[dependency-groups]
test = [
"pytest>=7.0.0",
"pytest-cov>=4.1.0",
]
lint = [
"black",
"ruff",
"mypy",
]
# Groups can include other groups
dev = [
{"include-group": "test"},
{"include-group": "lint"},
"pre-commit",
]

[tool.hatch.envs.test]
dependency-groups = ["test"]

[tool.hatch.envs.lint]
dependency-groups = ["lint"]

[tool.hatch.envs.dev]
dependency-groups = ["dev"]
Comment on lines +152 to +159
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can imagine that a completely reflected use-case would be preferable here. i don't know the current affairs reg. user documentation guidelines, maybe @lwasser has some advice.

```

The `dependency-groups` option specifies which PEP 735 dependency groups to include in the environment's dependencies. This is particularly useful for organizing related dependencies and including them in appropriate environments.

### Combining with Other Dependencies

Dependency groups can be combined with other dependency mechanisms:

```toml config-example
[project]
name = "my-app"
version = "0.1.0"
dependencies = [
"requests>=2.28.0",
]

[dependency-groups]
test = ["pytest>=7.0.0"]
docs = ["sphinx>=7.0.0"]

[tool.hatch.envs.test]
# Include the test dependency group
dependency-groups = ["test"]
# Add environment-specific dependencies
dependencies = [
"coverage[toml]>=7.0.0",
]
# Project dependencies will be included if not skip-install and in dev-mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't really get what this wants to tell me; it seems to be suited for a sentence or two outside the example snippet.

```

In this example, the test environment would include:
1. Project dependencies (`requests>=2.28.0`)
2. The test dependency group (`pytest>=7.0.0`)
3. Environment-specific dependencies (`coverage[toml]>=7.0.0`)

## Option overrides

You can modify options based on the conditions of different sources like [matrix variables](#matrix-variable-overrides) with the `overrides` table, using [dotted key](https://toml.io/en/v1.0.0#table) syntax for each declaration:
Expand Down
22 changes: 22 additions & 0 deletions docs/config/environment/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,28 @@ features = [
!!! note
Features/optional dependencies are also known as `extras` in other tools.

### Dependency Groups

You can include [PEP 735](https://peps.python.org/pep-0735/) dependency groups in your environments using the `dependency-groups` option:

```toml config-example
[dependency-groups]
test = [
"pytest>=7.0.0",
"pytest-cov>=4.1.0",
]

[tool.hatch.envs.test]
dependency-groups = [
"test",
]
```

Dependency groups provide a standardized way to organize related dependencies and can be shared across different build systems. See [advanced usage](advanced.md#dependency-groups) for more details on dependency group features like including other groups.

!!! note
Unlike features which affect how the project itself is installed, dependency groups are separate dependencies that are installed alongside the project.

Comment on lines +125 to +127
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the point here is to point out that dependency groups are no direct project dependencies, i think that aspect can be included in one of the prior paragraphs.

### Dev mode

By default, environments will always reflect the current state of your project on disk, for example, by installing it in editable mode in a Python environment. Set `dev-mode` to `false` to disable this behavior and have your project installed only upon creation of a new environment. From then on, you need to manage your project installation manually.
Expand Down
45 changes: 43 additions & 2 deletions src/hatch/env/plugin/interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -288,8 +288,8 @@ def dependencies_complex(self):

# Ensure these are checked last to speed up initial environment creation since
# they will already be installed along with the project
if (not self.skip_install and self.dev_mode) or self.features:
from hatch.utils.dep import get_complex_dependencies, get_complex_features
if (not self.skip_install and self.dev_mode) or self.features or self.dependency_groups:
from hatch.utils.dep import get_complex_dependencies, get_complex_dependency_group, get_complex_features

dependencies, optional_dependencies = self.app.project.get_dependencies()
dependencies_complex = get_complex_dependencies(dependencies)
Expand All @@ -308,6 +308,11 @@ def dependencies_complex(self):

all_dependencies_complex.extend(optional_dependencies_complex[feature].values())

for dependency_group in self.dependency_groups:
all_dependencies_complex.extend(
get_complex_dependency_group(self.app.project.dependency_groups, dependency_group)
)

return all_dependencies_complex

@cached_property
Expand Down Expand Up @@ -424,6 +429,42 @@ def features(self):

return sorted(all_features)

@cached_property
def dependency_groups(self):
from hatchling.metadata.utils import normalize_project_name

dependency_groups = self.config.get('dependency-groups', [])
if not isinstance(dependency_groups, list):
message = f'Field `tool.hatch.envs.{self.name}.dependency-groups` must be an array of strings'
raise TypeError(message)

all_dependency_groups = set()
for i, dependency_group in enumerate(dependency_groups, 1):
if not isinstance(dependency_group, str):
message = (
f'Dependency Group #{i} of field `tool.hatch.envs.{self.name}.dependency-groups` must be a string'
)
raise TypeError(message)

if not dependency_group:
message = f'Dependency Group #{i} of field `tool.hatch.envs.{self.name}.dependency-groups` cannot be an empty string'
raise ValueError(message)

normalized_dependency_group = normalize_project_name(dependency_group)
if (
not self.metadata.hatch.metadata.hook_config
and normalized_dependency_group not in self.app.project.dependency_groups
):
message = (
f'Dependency Group `{normalized_dependency_group}` of field `tool.hatch.envs.{self.name}.dependency-groups` is not '
f'defined in field `dependency-groups`'
)
raise ValueError(message)

all_dependency_groups.add(normalized_dependency_group)

return sorted(all_dependency_groups)

@cached_property
def description(self) -> str:
"""
Expand Down
34 changes: 33 additions & 1 deletion src/hatch/project/core.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
from __future__ import annotations

import re
from collections import defaultdict
from contextlib import contextmanager
from functools import cached_property
from typing import TYPE_CHECKING, Generator, cast
from typing import TYPE_CHECKING, Any, Generator, cast

from hatch.project.env import EnvironmentMetadata
from hatch.utils.fs import Path
Expand Down Expand Up @@ -103,6 +104,37 @@ def build_frontend(self) -> BuildFrontend:
def env_metadata(self) -> EnvironmentMetadata:
return EnvironmentMetadata(self.app.data_dir / 'env' / '.metadata', self.location)

@cached_property
def dependency_groups(self) -> dict[str, Any]:
"""
https://peps.python.org/pep-0735/
"""
from hatchling.metadata.utils import normalize_project_name

dependency_groups = self.raw_config.get('dependency-groups', {})

if not isinstance(dependency_groups, dict):
message = 'Field `dependency-groups` must be a table'
raise TypeError(message)

original_names = defaultdict(list)
normalized_groups = {}

for group_name, value in dependency_groups.items():
normed_group_name = normalize_project_name(group_name)
original_names[normed_group_name].append(group_name)
normalized_groups[normed_group_name] = value

errors = []
for normed_name, names in original_names.items():
if len(names) > 1:
errors.append(f"{normed_name} ({', '.join(names)})")
if errors:
msg = f"Field `dependency-groups` contains duplicate names: {', '.join(errors)}"
raise ValueError(msg)

return normalized_groups

def get_environment(self, env_name: str | None = None) -> EnvironmentInterface:
if env_name is None:
env_name = self.app.env
Expand Down
41 changes: 37 additions & 4 deletions src/hatch/utils/dep.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
from __future__ import annotations

from typing import TYPE_CHECKING
from typing import Any

from hatchling.metadata.utils import get_normalized_dependency
from packaging.requirements import Requirement

if TYPE_CHECKING:
from packaging.requirements import Requirement
from hatchling.metadata.utils import get_normalized_dependency, normalize_project_name


def normalize_marker_quoting(text: str) -> str:
Expand Down Expand Up @@ -52,3 +51,37 @@ def get_complex_features(features: dict[str, list[str]]) -> dict[str, dict[str,
}

return optional_dependencies_complex


def get_complex_dependency_group(
dependency_groups: dict[str, Any], group: str, past_groups: tuple[str, ...] = ()
) -> list[Requirement]:
if group in past_groups:
msg = f'Cyclic dependency group include: {group} -> {past_groups}'
raise ValueError(msg)

if group not in dependency_groups:
msg = f"Dependency group '{group}' not found"
raise LookupError(msg)

raw_group = dependency_groups[group]
if not isinstance(raw_group, list):
msg = f"Dependency group '{group}' is not a list"
raise TypeError(msg)

realized_group = []
for item in raw_group:
if isinstance(item, str):
realized_group.append(Requirement(item))
elif isinstance(item, dict):
if tuple(item.keys()) != ('include-group',):
msg = f'Invalid dependency group item: {item}'
raise ValueError(msg)

include_group = normalize_project_name(next(iter(item.values())))
realized_group.extend(get_complex_dependency_group(dependency_groups, include_group, (*past_groups, group)))
else:
msg = f'Invalid dependency group item: {item}'
raise TypeError(msg)

return realized_group
Loading
Loading