You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGES.md
+10-17
Original file line number
Diff line number
Diff line change
@@ -5553,7 +5553,7 @@ code_servers:
5553
5553
5554
5554
- [dagster-airbyte] Added an experimental function to automatically generate Airbyte assets from project YAML files. For more information, see the [dagster-airbyte docs](https://docs.dagster.io/_apidocs/libraries/dagster-airbyte).
5555
5555
- [dagster-airbyte] Added the forward_logs option to `AirbyteResource`, allowing users to disble forwarding of Airbyte logs to the compute log, which can be expensive for long-running syncs.
5556
-
- [dagster-airbyte] Added the ability to generate Airbyte assets for [basic normalization](https://docs.airbyte.com/understanding-airbyte/basic-normalization/#nesting) tables generated as part of a sync.
5556
+
- [dagster-airbyte] Added the ability to generate Airbyte assets for basic normalization tables generated as part of a sync.
5557
5557
5558
5558
### Documentation
5559
5559
@@ -5860,7 +5860,7 @@ to make sure the correct library version is installed.
5860
5860
5861
5861
### Documentation
5862
5862
5863
-
- The non-asset version of the Hacker News example, which lived inside `examples/hacker_news/`, has been removed, because it hadn’t received updates in a long time and had drifted from best practices. The asset version is still there and has an updated README. Check it out [here](https://github.com/dagster-io/dagster/tree/master/examples/hacker_news_assets)
5863
+
- The non-asset version of the Hacker News example, which lived inside `examples/hacker_news/`, has been removed, because it hadn’t received updates in a long time and had drifted from best practices. The asset version is still there and has an updated README.
5864
5864
5865
5865
## 0.15.6
5866
5866
@@ -6760,7 +6760,7 @@ asset_group = AssetGroup(
6760
6760
- [dagit] On the instance summary page, hovering over a recent run’s status dot shows a more helpful tooltip.
6761
6761
- [dagster-k8s] Improved performance of the `k8s_job_executor` for runs with many user logs
6762
6762
- [dagster-k8s] When using the `dagster-k8s/config` tag to configure Dagster Kubernetes pods, the tags can now accept any valid Kubernetes config, and can be written in either snake case (`node_selector_terms`) or camel case (`nodeSelectorTerms`). See [the docs](https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment) for more information.
6763
-
- [dagster-aws] You can now [set secrets on the `EcsRunLauncher` using the same syntax](https://legacy-versioned-docs.dagster.dagster-docs.io/0.14.4/deployment/guides/aws#secrets-management-in-ecs) that you use to set secrets in the ECS API.
6763
+
- [dagster-aws] You can now set secrets on the `EcsRunLauncher` using the same syntax that you use to set secrets in the ECS API.
6764
6764
- [dagster-aws] The `EcsRunLauncher` now attempts to reuse task definitions instead of registering a new task definition for every run.
6765
6765
- [dagster-aws] The `EcsRunLauncher` now raises the underlying ECS API failure if it cannot successfully start a task.
6766
6766
@@ -6820,7 +6820,7 @@ asset_group = AssetGroup(
6820
6820
- Renamed the filter class from `PipelineRunsFilter` => `RunsFilter`.
6821
6821
- Assets can now be directly invoked for unit testing.
6822
6822
- [dagster-dbt] `load_assets_from_dbt_project` will now attach schema information to the generated assets if it is available in the dbt project (`schema.yml`).
6823
-
- [examples] Added an [example](https://github.com/dagster-io/dagster/tree/master/examples/modern_data_stack_assets) that demonstrates using Software Defined Assets with Airbyte, dbt, and custom Python.
6823
+
- [examples] Added an example that demonstrates using Software Defined Assets with Airbyte, dbt, and custom Python.
6824
6824
- The default io manager used in the `AssetGroup` api is now the `fs_asset_io_manager`.
6825
6825
- It's now possible to build a job where partitioned assets depend on partitioned assets that are maintained outside the job, and for those upstream partitions to show up on the context in the op and IOManager load_input function.
6826
6826
- `SourceAsset`s can now be partitioned, by setting the `partitions_def` argument.
@@ -6932,7 +6932,7 @@ asset_group = AssetGroup(
6932
6932
```
6933
6933
6934
6934
- A new Airbyte integration [(dagster-airbyte)](https://docs.dagster.io/_apidocs/libraries/dagster-airbyte#airbyte-dagster-airbyte) allows you to kick off and monitor [Airbyte](https://airbyte.com/) syncs from within Dagster. The original contribution from @airbytehq’s own @marcosmarxm includes a [resource implementation](https://docs.dagster.io/_apidocs/libraries/dagster-airbyte#resources) as well as a [pre-built op](https://docs.dagster.io/_apidocs/libraries/dagster-airbyte#ops) for this purpose, and we’ve extended this library to support [software-defined asset](https://docs.dagster.io/_apidocs/libraries/dagster-airbyte#assets) use cases as well. Regardless of which interface you use, Dagster will automatically capture the Airbyte log output (in the compute logs for the relevant steps) and track the created tables over time (via AssetMaterializations).
6935
-
- The [ECSRunLauncher](https://docs.dagster.io/deployment/guides/ecs) (introduced in Dagster 0.11.15) is no longer considered experimental. You can bootstrap your own Dagster deployment on ECS using our [docker compose example](https://github.com/dagster-io/dagster/tree/master/examples/deploy_ecs) or you can use it in conjunction with a [managed Dagster Cloud deployment](https://docs.dagster.cloud/agents/ecs/setup). Since its introduction, we’ve added the ability to customize Fargate container memory and CPU, mount secrets from AWS SecretsManager, and run with a variety of AWS networking configurations. Join us in [#dagster-ecs](https://dagster.slack.com/archives/C014UDS8LAV) in Slack!
6935
+
- The [ECSRunLauncher](https://docs.dagster.io/deployment/guides/ecs) (introduced in Dagster 0.11.15) is no longer considered experimental. You can bootstrap your own Dagster deployment on ECS using our [docker compose example](https://github.com/dagster-io/dagster/tree/master/examples/deploy_ecs) or you can use it in conjunction with a [managed Dagster Cloud deployment](https://docs.dagster.io/dagster-plus/deployment/deployment-types/hybrid/amazon-ecs/). Since its introduction, we’ve added the ability to customize Fargate container memory and CPU, mount secrets from AWS SecretsManager, and run with a variety of AWS networking configurations. Join us in [#dagster-ecs](https://dagster.slack.com/archives/C014UDS8LAV) in Slack!
6936
6936
- [Helm] The default liveness and startup probes for Dagit and user deployments have been replaced with readiness probes. The liveness and startup probe for the Daemon has been removed. We observed and heard from users that under load, Dagit could fail the liveness probe which would result in the pod restarting. With the new readiness probe, the pod will not restart but will stop serving new traffic until it recovers. If you experience issues with any of the probe changes, you can revert to the old behavior by specifying liveness and startup probes in your Helm values (and reach out via an issue or Slack).
6937
6937
6938
6938
### Breaking Changes and Deprecations
@@ -7712,7 +7712,7 @@ execution:
7712
7712
### New
7713
7713
7714
7714
- [dagster-k8s] When launching a pipeline using the K8sRunLauncher or k8s_job_executor, you can know specify a list of volumes to be mounted in the created pod. See the [API docs](https://docs.dagster.io/_apidocs/libraries/dagster-k8s#dagster_k8s.K8sRunLauncher) for for information.
7715
-
- [dagster-k8s] When specifying a list of environment variables to be included in a pod using [custom configuration](https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#solid-or-pipeline-kubernetes-configuration), you can now specify the full set of parameters allowed by a [V1EnvVar](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#envvar-v1-core) in Kubernetes.
7715
+
- [dagster-k8s] When specifying a list of environment variables to be included in a pod using [custom configuration](https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#solid-or-pipeline-kubernetes-configuration), you can now specify the full set of parameters allowed by a [V1EnvVar](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#envvar-v1-core) in Kubernetes.
7716
7716
7717
7717
### Bugfixes
7718
7718
@@ -7871,8 +7871,7 @@ run_coordinator:
7871
7871
7872
7872
- When a solid, pipeline, schedule, etc. description or event metadata entry contains a
7873
7873
markdown-formatted table, that table is now rendered in Dagit with better spacing between elements.
- The hacker-news example now includes instructions
7876
7875
on how to deploy the repository in a Kubernetes cluster using the Dagster Helm chart.
7877
7876
- [dagster-dbt] The `dbt_cli_resource` now supports the `dbt source snapshot-freshness` command
7878
7877
(thanks @emilyhawkins-drizly!)
@@ -8353,7 +8352,7 @@ def my_root_manager(_):
8353
8352
8354
8353
### Experimental
8355
8354
8356
-
- Added an example of run attribution via a [custom run coordinator](https://github.com/dagster-io/dagster/tree/master/examples/run_attribution_example), which reads a user’s email from HTTP headers on the Dagster GraphQL server and attaches the email as a run tag. Custom run coordinator are also now specifiable in the Helm chart, under `queuedRunCoordinator`. See the [docs](https://docs.dagster.io/master/guides/dagster/run-attribution) for more information on setup.
8355
+
- Added an example of run attribution via a custom run coordinator, which reads a user’s email from HTTP headers on the Dagster GraphQL server and attaches the email as a run tag. Custom run coordinator are also now specifiable in the Helm chart, under `queuedRunCoordinator`. See the [docs](https://docs.dagster.io/master/guides/dagster/run-attribution) for more information on setup.
8357
8356
- `RetryPolicy` now supports backoff and jitter settings, to allow for modulating the `delay` as a function of attempt number and randomness.
8358
8357
8359
8358
### Documentation
@@ -10138,7 +10137,7 @@ opt_in:
10138
10137
- An initial version of the CeleryDockerExecutor was added (thanks @mrdrprofuroboros!). The celery
10139
10138
workers will launch tasks in docker containers.
10140
10139
- **Experimental:** Great Expectations integration is currently under development in the new library
10141
-
dagster-ge. Example usage can be found [here](https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-ge/dagster_ge/examples/ge_demo.py)
10140
+
dagster-ge.
10142
10141
10143
10142
## 0.8.5
10144
10143
@@ -11523,14 +11522,10 @@ Thank you to all of the community contributors to this release!! In alphabetical
11523
11522
11524
11523
- Adds a `type_check` parameter to `PythonObjectType`, `as_dagster_type`, and `@as_dagster_type` to
11525
11524
enable custom type checks in place of default `isinstance` checks.
0 commit comments