Skip to content

Commit 76408dc

Browse files
authored
[dg] Update uv integration guide (BUILD-973) (#29181)
## Summary & Motivation - Rename the "uv integration" guide to "Python environment management and uv integration" - Update the content for accuracy/new APIs Note that this will go through another iteration, I'm working on a longer guide that explains `dg`'s process/environment architecture in detail. I think this guide will be folded into that when it's done. ## How I Tested These Changes Existing test suite.
1 parent 07fa58f commit 76408dc

File tree

3 files changed

+37
-42
lines changed

3 files changed

+37
-42
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
title: 'Python environment management and `uv` integration'
3+
sidebar_position: 500
4+
---
5+
6+
import Preview from '@site/docs/partials/\_Preview.md';
7+
8+
<Preview />
9+
10+
:::note
11+
12+
We are working on streamlining the configuration of the `uv` integration, and it
13+
is likely to change in the next few releases.
14+
15+
:::
16+
17+
Many `dg` commands need to spawn subprocesses. For example, when you run `dg list plugins` in a project directory, `dg` does not perform an in-process inspection of the environment in which `dg` itself is running. Instead, it resolves a Python environment for the in-scope project and spawns a subprocess using the `dagster-components` executable (an internal API) in that environment. The output you see on the command line represents the plugins available in the resolved environment, which is not necessarily the same as the environment in which `dg` itself is running.
18+
19+
The Python environment used for a project is determined by the `tool.dg.project.python_environment` setting in the project `pyproject.toml` file. This setting is either:
20+
21+
- `active` (the default), which means that `dg` will use the currently active Python environment.
22+
- `persistent_uv` (not default, but set in dg-scaffolded projects by default), which means that `dg` will use a dedicated Python environment found in `<project_root>/.venv` that is created and managed by `uv`.
23+
24+
If the setting is `active`, then `dg` does nothing fancy. It is up to the user to manage their own Python environments. If individual project-scoped environments are desired, the user must create them, and ensure the appropriate one is activated when running `dg` commands against that project.
25+
26+
If the setting is `persistent_uv`, then `dg` will ignore any activated virtual environments. Subprocesses will be launched using `uv run`, which handles resolution of the project-scoped `.venv` environment and ensures subprocesses are launched in that environment. It also ensures that the environment is up to date with the project's specified dependencies before launching a subprocess.
27+
28+
The `persistent_uv` setting is intended to ease development friction by removing the burden of juggling virtual environments from the user. It is our recommended configuration. However, we are still developing its functionality, and some users may prefer to totally opt out of any environment management "magic". These users should make sure all their projects use `tool.dg.project.python_environment = "active"`.
29+
30+
:::info
31+
32+
Disabling `uv` integration by setting `tool.dg.project.python_environment
33+
= "active"` will currently disable the `dg cache`. This will make some
34+
operations noticeably slower.
35+
36+
:::

docs/docs/guides/labs/dg/scaffolding-a-project.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,4 @@ The `dg init` command creates a directory with a standard Python package structu
3232
- `pyproject.toml` is a standard Python package configuration file. In addition
3333
to the regular Python package metadata, it contains a `tool.dg` section
3434
for `dg`-specific settings.
35-
- `uv.lock` is the [lockfile](https://docs.astral.sh/uv/concepts/projects/layout/#the-lockfile) for the Python package manager [`uv`](https://docs.astral.sh/uv/). `dg` projects use `uv` by default. For more information, see [`uv` integration](/guides/labs/dg/uv-integration).
35+
- `uv.lock` is the [lockfile](https://docs.astral.sh/uv/concepts/projects/layout/#the-lockfile) for the Python package manager [`uv`](https://docs.astral.sh/uv/). `dg` projects use `uv` by default. For more information, see [`uv` integration](/guides/labs/dg/python-environment-management-and-uv-integration).

docs/docs/guides/labs/dg/uv-integration.md

-41
This file was deleted.

0 commit comments

Comments
 (0)