Skip to content

Commit 0feae7d

Browse files
maximearmstrongbrentjericho
authored andcommitted
[docs] Update lifecycle warnings for integrations (dagster-io#27635)
## Summary & Motivation Update warnings for all integrations changed in this [PR stack](dagster-io#26865) for the API Lifecycle project. ## How I Tested These Changes BK and docs preview
1 parent 7c67fb4 commit 0feae7d

34 files changed

+99
-71
lines changed

docs/docs/guides/build/external-pipelines/aws-emr-containers-pipeline.md

+4
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ description: "Learn to integrate Dagster Pipes with AWS EMR Containers to launch
44
sidebar_position: 300
55
---
66

7+
import Preview from '../../../partials/\_Preview.md';
8+
9+
<Preview />
10+
711
This tutorial gives a short overview on how to use [Dagster Pipes](/guides/build/external-pipelines/) with [AWS EMR on EKS](https://aws.amazon.com/emr/features/eks/) (the corresponding AWS API is called `emr-containers`).
812

913
The [dagster-aws](/api/python-api/libraries/dagster-aws) integration library provides the <PyObject section="libraries" object="pipes.PipesEMRContainersClient" module="dagster_aws" /> resource, which can be used to launch EMR jobs from Dagster assets and ops. Dagster can receive regular events such as logs, asset checks, or asset materializations from jobs launched with this client. Using it requires minimal code changes to your EMR jobs.

docs/docs/integrations/libraries/airbyte/airbyte-cloud.md

+2-4
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,9 @@ sidebar_custom_props:
1919
logo: images/integrations/airbyte.svg
2020
---
2121

22-
:::note
22+
import Beta from '../../../partials/\_Beta.md';
2323

24-
This integration is currently in **beta**.
25-
26-
:::
24+
<Beta />
2725

2826
This guide provides instructions for using Dagster with Airbyte Cloud using the `dagster-airbyte` library. Your Airbyte Cloud connection tables can be represented as assets in the Dagster asset graph, allowing you to track lineage and dependencies between Airbyte Cloud assets and data assets you are already modeling in Dagster. You can also use Dagster to orchestrate Airbyte Cloud connections, allowing you to trigger syncs for these on a cadence or based on upstream data changes.
2927

docs/docs/integrations/libraries/airlift/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ sidebar_custom_props:
1919
logo: images/integrations/airflow.svg
2020
---
2121

22+
import Beta from '../../../partials/\_Beta.md';
23+
24+
<Beta />
25+
2226
Airlift is a toolkit for integrating Dagster and Airflow. Using `dagster-airflift`, you can:
2327

2428
- Observe Airflow instances from within Dagster

docs/docs/integrations/libraries/aws/cloudwatch.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/aws-cloudwatch.svg
1919
---
2020

21+
import Deprecated from '../../../partials/\_Deprecated.md';
22+
23+
<Deprecated />
24+
2125
This integration allows you to send Dagster logs to AWS CloudWatch, enabling centralized logging and monitoring of your Dagster jobs. By using AWS CloudWatch, you can take advantage of its powerful log management features, such as real-time log monitoring, log retention policies, and alerting capabilities.
2226

2327
Using this integration, you can configure your Dagster jobs to log directly to AWS CloudWatch, making it easier to track and debug your workflows. This is particularly useful for production environments where centralized logging is essential for maintaining observability and operational efficiency.

docs/docs/integrations/libraries/aws/ecr.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/aws-ecr.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
This integration allows you to connect to AWS Elastic Container Registry (ECR). It provides resources to interact with AWS ECR, enabling you to manage your container images.
2226

2327
Using this integration, you can seamlessly integrate AWS ECR into your Dagster pipelines, making it easier to manage and deploy containerized applications.

docs/docs/integrations/libraries/aws/secretsmanager.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/aws-secretsmanager.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
This integration allows you to manage, retrieve, and rotate credentials, API keys, and other secrets using [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/).
2226

2327
### Installation

docs/docs/integrations/libraries/aws/ssm.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/aws-ssm.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
The Dagster AWS Systems Manager (SSM) Parameter Store integration allows you to manage and retrieve parameters stored in AWS SSM Parameter Store directly within your Dagster pipelines. This integration provides resources to fetch parameters by name, tags, or paths, and optionally set them as environment variables for your operations.
2226

2327
### Installation

docs/docs/integrations/libraries/datadog.md

+4
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ sidebar_custom_props:
1919
logo: images/integrations/datadog.svg
2020
---
2121

22+
import Beta from '../../partials/\_Beta.md';
23+
24+
<Beta />
25+
2226
While Dagster provides comprehensive monitoring and observability of the pipelines it orchestrates, many teams look to centralize all their monitoring across apps, processes and infrastructure using Datadog's 'Cloud Monitoring as a Service'. The `dagster-datadog` integration allows you to publish metrics to Datadog from within Dagster ops.
2327

2428
### Installation

docs/docs/integrations/libraries/dbt/dbt-cloud.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/dbt/dbt.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
Dagster allows you to run dbt Cloud jobs alongside other technologies. You can schedule them to run as a step in a larger pipeline and manage them as a data asset.
2226

2327
### Installation

docs/docs/integrations/libraries/dbt/reference.md

-4
Original file line numberDiff line numberDiff line change
@@ -434,8 +434,6 @@ Dagster supports fetching additional metadata at dbt execution time to attach as
434434

435435
:::note
436436

437-
**Emitting row count data for dbt models is currently an experimental feature.**
438-
439437
To use this feature, you'll need to be on at least `dagster>=0.17.6` and `dagster-dbt>=0.23.6`.
440438

441439
:::
@@ -452,8 +450,6 @@ Once your dbt models have been materialized, you can view the row count data in
452450

453451
:::note
454452

455-
**Emitting column-level metadata for dbt models is currently an experimental feature.**
456-
457453
To use this feature, you'll need to be on at least `dagster>=1.8.0` and `dagster-dbt>=0.24.0`.
458454

459455
:::

docs/docs/integrations/libraries/dlt/index.md

-6
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,6 @@ sidebar_custom_props:
2121

2222
This integration allows you to use [dlt](https://dlthub.com/) to easily ingest and replicate data between systems through Dagster.
2323

24-
:::note
25-
26-
This integration is currently **experimental**.
27-
28-
:::
29-
3024
### Installation
3125

3226
```bash

docs/docs/integrations/libraries/dlt/using-dlt-with-dagster.md

+1-7
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,6 @@ title: "Using dlt with Dagster"
33
description: Ingest data with ease using Dagster and dlt
44
---
55

6-
:::
7-
8-
This feature is considered **experimental**
9-
10-
:::
11-
126
The [data load tool (dlt)](https://dlthub.com/) open-source library defines a standardized approach for creating data pipelines that load often messy data sources into well-structured data sets. It offers many advanced features, such as:
137

148
- Handling connection secrets
@@ -244,7 +238,7 @@ For example, let's say we have defined a set of dlt assets named `thinkific_asse
244238
245239
### Using partitions in your dlt assets
246240
247-
While still an experimental feature, it is possible to use partitions within your dlt assets. However, it should be noted that this may result in concurrency related issues as state is managed by dlt. For this reason, it is recommended to set concurrency limits for your partitioned dlt assets. See the [Limiting concurrency in data pipelines](/guides/operate/managing-concurrency) guide for more details.
241+
It is possible to use partitions within your dlt assets. However, it should be noted that this may result in concurrency related issues as state is managed by dlt. For this reason, it is recommended to set concurrency limits for your partitioned dlt assets. See the [Limiting concurrency in data pipelines](/guides/operate/managing-concurrency) guide for more details.
248242
249243
That said, here is an example of using static named partitions from a dlt source.
250244

docs/docs/integrations/libraries/embedded-elt.md

-5
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,6 @@ This package includes two integrations:
2424
- [Sling](https://slingdata.io) provides a simple way to sync data between databases and file systems.
2525
- [data Load Tool (dlt)](https://dlthub.com) easily loads data from external systems and APIs.
2626

27-
:::note
28-
29-
This integration is currently **experimental**.
30-
31-
:::
3227

3328
## Installation
3429

docs/docs/integrations/libraries/fivetran.md

+2-4
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,9 @@ sidebar_custom_props:
1919
logo: images/integrations/fivetran.svg
2020
---
2121

22-
:::note
22+
import Beta from '../../partials/\_Beta.md';
2323

24-
This integration is currently in **beta**.
25-
26-
:::
24+
<Beta />
2725

2826
This guide provides instructions for using Dagster with Fivetran using the `dagster-fivetran` library. Your Fivetran connector tables can be represented as assets in the Dagster asset graph, allowing you to track lineage and dependencies between Fivetran assets and data assets you are already modeling in Dagster. You can also use Dagster to orchestrate Fivetran connectors, allowing you to trigger syncs for these on a cadence or based on upstream data changes.
2927

docs/docs/integrations/libraries/gcp/bigquery/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/gcp-bigquery.svg
1919
---
2020

21+
import Beta from '../../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
The Google Cloud Platform BigQuery integration allows data engineers to easily query and store data in the BigQuery data warehouse through the use of the `BigQueryResource`.
2226

2327
### Installation

docs/docs/integrations/libraries/gcp/bigquery/using-bigquery-with-dagster.md

+4
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ description: Store your Dagster assets in BigQuery
44
sidebar_position: 100
55
---
66

7+
import Beta from '../../../../partials/\_Beta.md';
8+
9+
<Beta />
10+
711
This tutorial focuses on creating and interacting with BigQuery tables using Dagster's [asset definitions](/guides/build/assets/defining-assets).
812

913
The `dagster-gcp` library provides two ways to interact with BigQuery tables:

docs/docs/integrations/libraries/gcp/dataproc.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/gcp-dataproc.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
Using this integration, you can manage and interact with Google Cloud Platform's Dataproc service directly from Dagster. This integration allows you to create, manage, and delete Dataproc clusters, and submit and monitor jobs on these clusters.
2226

2327
### Installation

docs/docs/integrations/libraries/github.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/github.svg
1919
---
2020

21+
import Deprecated from '../../partials/\_Deprecated.md';
22+
23+
<Deprecated />
24+
2125
This library provides an integration with _[GitHub Apps](https://docs.github.com/en/developers/apps/getting-started-with-apps/about-apps)_ by providing a thin wrapper on the GitHub v4 GraphQL API. This allows for automating operations within your GitHub repositories and with the tighter permissions scopes that GitHub Apps allow for vs using a personal token.
2226

2327
### Installation

docs/docs/integrations/libraries/jupyter/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ sidebar_custom_props:
1919
logo: images/integrations/jupyter.svg
2020
---
2121

22+
import Beta from '../../../partials/\_Beta.md';
23+
24+
<Beta />
25+
2226
Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.
2327

2428
Using the Dagstermill library enables you to:

docs/docs/integrations/libraries/looker/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ sidebar_custom_props:
1919
logo: images/integrations/looker.svg
2020
---
2121

22+
import Beta from '../../../partials/\_Beta.md';
23+
24+
<Beta />
25+
2226
Dagster allows you to represent your Looker project as assets, alongside other your other technologies like dbt and Sling. This allows you to see how your Looker assets are connected to your other data assets, and how changes to other data assets might impact your Looker project.
2327

2428
### Installation

docs/docs/integrations/libraries/looker/using-looker-with-dagster.md

+2-5
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,9 @@ title: "Using Looker with Dagster"
33
description: Represent your Looker assets in Dagster
44
---
55

6-
:::
7-
8-
This feature is considered **experimental**
9-
10-
:::
6+
import Beta from '../../../partials/\_Beta.md';
117

8+
<Beta />
129

1310
This guide provides instructions for using Dagster with Looker using the `dagster-looker` library. Your Looker assets, such as views, explores, and dashboards, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Looker assets. You can also use Dagster to orchestrate Looker PDTs, allowing you to trigger refreshes of these materialized tables on a cadence or based on upstream data changes.
1411

docs/docs/integrations/libraries/openai/using-openai-with-dagster.md

-6
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,6 @@ title: "OpenAI & Dagster"
33
description: "The dagster-openai library provides the ability to build OpenAI pipelines with Dagster and log OpenAI API usage metadata in Dagster Insights."
44
---
55

6-
:::note
7-
8-
This feature is considered **experimental**
9-
10-
:::
11-
126
The `dagster-openai` library allows you to build OpenAI pipelines with Dagster and log OpenAI API usage metadata in [Dagster Insights](/dagster-plus/features/insights).
137

148
Using this library's <PyObject section="libraries" module="dagster_openai" object="OpenAIResource" />, you can easily interact with the [OpenAI REST API](https://platform.openai.com/docs/introduction) via the [OpenAI Python API](https://github.com/openai/openai-python).

docs/docs/integrations/libraries/pandas/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/pandas.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
Perform data validation, emit summary statistics, and enable reliable DataFrame serialization/deserialization. The dagster_pandas library provides you with the utilities for implementing validation on Pandas DataFrames. The Dagster type system generates documentation of your DataFrame constraints and makes it accessible in the Dagster UI.
2226

2327
### Installation

docs/docs/integrations/libraries/pandas/using-pandas-with-dagster.md

+4
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@ description: "The dagster-pandas library provides the ability to perform data va
55

66
# Pandas & Dagster
77

8+
import Beta from '../../../partials/\_Beta.md';
9+
10+
<Beta />
11+
812
:::note
913

1014
This page describes the `dagster-pandas` library, which is used for performing data validation. To simply use pandas with Dagster, start with the [Dagster Quickstart](/getting-started/quickstart)

docs/docs/integrations/libraries/pandera/index.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/pandera.svg
1919
---
2020

21+
import Beta from '../../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
The `dagster-pandera` integration library provides an API for generating Dagster Types from [Pandera DataFrame schemas](https://pandera.readthedocs.io/en/stable/dataframe_schemas.html).
2226

2327
Like all Dagster types, Dagster-Pandera-generated types can be used to annotate op inputs and outputs. This provides runtime type-checking with rich error reporting and allows Dagster UI to display information about a DataFrame's structure.

docs/docs/integrations/libraries/pandera/using-pandera-with-dagster.md

+4
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@ description: Generate Dagster types for dataframes with Pandera.
55

66
# Pandera & Dagster
77

8+
import Beta from '../../../partials/\_Beta.md';
9+
10+
<Beta />
11+
812
The `dagster-pandera` integration library provides an API for generating [Dagster Types](/api/python-api/types) from [Pandera](https://github.com/pandera-dev/pandera) dataframe schemas. Like all Dagster types, `dagster-pandera`-generated types can be used to annotate [op](/guides/build/ops) inputs and outputs.
913

1014
Using Pandera with Dagster allows you to:

docs/docs/integrations/libraries/powerbi/index.md

+3-5
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,11 @@ sidebar_custom_props:
1717
logo: images/integrations/powerbi.svg
1818
---
1919

20-
This guide provides instructions for using Dagster with Power BI using the `dagster-powerbi` library. Your Power BI assets, such as semantic models, data sources, reports, and dashboards, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Power BI assets and upstream data assets you are already modeling in Dagster. You can also use Dagster to orchestrate Power BI semantic models, allowing you to trigger refreshes of these models on a cadence or based on upstream data changes.
21-
22-
:::note
20+
import Beta from '../../../partials/\_Beta.md';
2321

24-
This integration is currently **experimental**.
22+
<Beta />
2523

26-
:::
24+
This guide provides instructions for using Dagster with Power BI using the `dagster-powerbi` library. Your Power BI assets, such as semantic models, data sources, reports, and dashboards, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Power BI assets and upstream data assets you are already modeling in Dagster. You can also use Dagster to orchestrate Power BI semantic models, allowing you to trigger refreshes of these models on a cadence or based on upstream data changes.
2725

2826

2927
## What you'll learn

docs/docs/integrations/libraries/powerbi/using-powerbi-with-dagster.md

+2-4
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,9 @@ title: "Using Power BI with Dagster"
33
description: Represent your Power BI assets in Dagster
44
---
55

6-
:::note
6+
import Beta from '../../../partials/\_Beta.md';
77

8-
This feature is considered **experimental**
9-
10-
:::
8+
<Beta />
119

1210
This guide provides instructions for using Dagster with Power BI using the [`dagster-powerbi`](/api/python-api/libraries/dagster-powerbi) library. Your Power BI assets, such as semantic models, data sources, reports, and dashboards, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Power BI assets and upstream data assets you are already modeling in Dagster. You can also use Dagster to orchestrate Power BI semantic models, allowing you to trigger refreshes of these models on a cadence or based on upstream data changes.
1311

docs/docs/integrations/libraries/prometheus.md

+4
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ sidebar_custom_props:
1818
logo: images/integrations/prometheus.svg
1919
---
2020

21+
import Beta from '../../partials/\_Beta.md';
22+
23+
<Beta />
24+
2125
This integration allows you to push metrics to the Prometheus gateway from within a Dagster pipeline.
2226

2327
### Installation

docs/docs/integrations/libraries/sigma/index.md

+3-6
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,11 @@ sidebar_custom_props:
1717
logo: images/integrations/sigma.svg
1818
---
1919

20-
This guide provides instructions for using Dagster with Sigma using the `dagster-sigma` library. Your Sigma assets, including datasets and workbooks, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Sigma assets and upstream data assets you are already modeling in Dagster.
21-
22-
:::note
20+
import Beta from '../../../partials/\_Beta.md';
2321

24-
This integration is currently **experimental**.
25-
26-
:::
22+
<Beta />
2723

24+
This guide provides instructions for using Dagster with Sigma using the `dagster-sigma` library. Your Sigma assets, including datasets and workbooks, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Sigma assets and upstream data assets you are already modeling in Dagster.
2825

2926
## What you'll learn
3027

docs/docs/integrations/libraries/sigma/using-sigma-with-dagster.md

+2-4
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,9 @@ title: "Using Sigma with Dagster"
33
description: Represent your Sigma assets in Dagster
44
---
55

6-
:::note
6+
import Beta from '../../../partials/\_Beta.md';
77

8-
This feature is considered **experimental**
9-
10-
:::
8+
<Beta />
119

1210
This guide provides instructions for using Dagster with Sigma using the [`dagster-sigma`](/api/python-api/libraries/dagster-sigma) library. Your Sigma assets, including datasets and workbooks, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Sigma assets and upstream data assets you are already modeling in Dagster.
1311

0 commit comments

Comments
 (0)