Skip to content

Commit 43722d7

Browse files
authored
DOC-1038 Add dg CLI API docs (#29193)
## Summary & Motivation Adds `dg` CLI docs to API docs and lightly restructures all API docs: * Adds Overview section * Creates sections for Dagster core and Dagster libraries docs for easier navigation and maintenance -- these sections map to the module names (`dagster` and `libraries`). * Moves each section's index file to Sphinx source for easier maintenance * Adds a page of CLI documentation to the Labs section ### TODO - [x] Fix broken links - [x] Add redirects ## Before <img width="1185" alt="Screenshot 2025-04-10 at 5 24 05 PM" src="https://github.com/user-attachments/assets/d42863b9-a131-4930-b4c4-bee5fe219461" /> ## After <img width="1213" alt="Screenshot 2025-04-10 at 5 26 34 PM" src="https://github.com/user-attachments/assets/47810eb1-ddfc-4c02-874b-f19566f5f7ad" /> ### After with sections collapsed <img width="1211" alt="Screenshot 2025-04-10 at 5 26 26 PM" src="https://github.com/user-attachments/assets/c818b4bd-3271-4749-bbf8-8b22b5b5ca1c" /> ## How I Tested These Changes ## Changelog > Insert changelog entry or delete this section. --------- Signed-off-by: nikki everett <[email protected]>
1 parent 3642d11 commit 43722d7

File tree

151 files changed

+608
-492
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

151 files changed

+608
-492
lines changed

docs/CONTRIBUTING.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,20 @@ When you make changes to the API, you may need to do some or all of the followin
4848
* Add or update docstrings in Python files
4949
* Update reStructuredText files to reference new modules, classes, or methods
5050

51-
In any case, you will need to follow the [reStructuredText syntax](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html).
51+
#### Formatting API documentation
52+
53+
When updating API docstrings or reStructuredText files, you will need to follow the [reStructuredText syntax](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html). Headings are formatted as follows:
54+
55+
```
56+
H1 heading
57+
==========
58+
59+
H2 heading
60+
----------
61+
62+
H3 heading
63+
^^^^^^^^^^
64+
```
5265

5366
## Formatting
5467

docs/docs/api/api-lifecycle/index.md renamed to docs/docs/api/api-lifecycle/api-lifecycle-stages.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
2-
sidebar_class_name: hidden
32
title: API lifecycle stages
3+
sidebar_position: 100
44
---
55

66
This reference guide outlines the different stages in the lifecycle of Dagster APIs, from preview to deprecation. Understanding these stages helps developers make informed decisions about which APIs to use in their projects.

docs/docs/api/api-lifecycle/filtering-api-lifecycle-warnings.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
title: Filtering API lifecycle warnings
3+
sidebar_position: 200
34
---
45

56
By default, Dagster logs warnings when APIs marked as preview, beta, superseded and deprecated are used in your code base. These warnings can be filtered out using the `warnings` module available by default in Python.
Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,2 @@
11
**/*.mdx
2-
!index.mdx
32
!external-assets-rest-api.mdx

docs/docs/api/python-api/external-assets-rest-api.mdx renamed to docs/docs/api/dagster/external-assets-rest-api.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -727,4 +727,4 @@ curl --request POST \
727727

728728
## Instance API
729729

730-
Refer to the [External assets instance API doc](/api/python-api/external-assets-instance-api) for information on the instance API.
730+
Refer to the [External assets instance API doc](/api/dagster/external-assets-instance-api) for information on the instance API.

docs/docs/api/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22
title: API reference
33
---
44

5-
These docs aim to cover the entire public surface of the core dagster APIs, as well as public APIs from all provided libraries.
5+
These docs aim to cover the entire public surface of the core dagster APIs, as well as the dagster-dg API, and public APIs from all provided libraries.
66

77
Dagster follows [semantic versioning](https://semver.org/). We attempt to isolate breaking changes to the public APIs to minor versions on a roughly 12-week cadence, and will announce deprecations in Slack and in the release notes to patch versions on a roughly weekly cadence.

docs/docs/api/libraries/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
**/*.mdx

docs/docs/api/python-api/index.mdx

Lines changed: 0 additions & 8 deletions
This file was deleted.

docs/docs/dagster-plus/deployment/management/managing-compute-logs-and-error-messages.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@ Depending on your organization's needs, you may want to retain these logs in you
1919

2020
## Modifying compute log storage
2121

22-
Dagster's compute logs are handled by the configured [`ComputeLogManager`](/api/python-api/internals#compute-log-manager). By default, Dagster+ utilizes the `CloudComputeLogManager` which stores logs in a Dagster+-managed Amazon S3 bucket, but you can customize this behavior to store logs in a destination of your choice.
22+
Dagster's compute logs are handled by the configured [`ComputeLogManager`](/api/dagster/internals#compute-log-manager). By default, Dagster+ utilizes the `CloudComputeLogManager` which stores logs in a Dagster+-managed Amazon S3 bucket, but you can customize this behavior to store logs in a destination of your choice.
2323

2424
### Writing to your own S3 bucket
2525

26-
If using the Kubernetes agent, you can instead forward logs to your own S3 bucket by using the [`S3ComputeLogManager`](/api/python-api/libraries/dagster-aws#dagster_aws.s3.S3ComputeLogManager).
26+
If using the Kubernetes agent, you can instead forward logs to your own S3 bucket by using the [`S3ComputeLogManager`](/api/libraries/dagster-aws#dagster_aws.s3.S3ComputeLogManager).
2727

2828
You can configure the `S3ComputeLogManager` in your [`dagster.yaml` file](/dagster-plus/deployment/management/settings/customizing-agent-settings):
2929

docs/docs/dagster-plus/deployment/management/settings/customizing-agent-settings.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ compute_logs:
9898

9999
## Writing compute logs to AWS S3
100100

101-
{/* /api/python-api/libraries/dagster-aws#dagster_aws.s3.S3ComputeLogManager */}
101+
{/* /api/libraries/dagster-aws#dagster_aws.s3.S3ComputeLogManager */}
102102
You can write compute logs to an AWS S3 bucket by configuring the <PyObject section="libraries" module="dagster_aws" object="s3.S3ComputeLogManager" /> module.
103103

104104
You are also able to stream partial compute log files by configuring the log upload interval (in seconds) using the `upload_interval` parameter.

0 commit comments

Comments
 (0)