-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
What's the issue?
When trying to add tags to assets loaded via the DbtProjectComponent that are strings but can be coerced to other types (int, dict), the code location fails to load.
dagster._core.errors.DagsterInvalidDefinitionError: Tag values must be strings, got type <class 'dict'> at key ecs/task_overrides".
or
dagster._core.errors.DagsterInvalidDefinitionError: Tag values must be strings, got type <class 'int'> at key ecs/cpu.
This is blocking me from using the DbtProjectComponent because the default resources for ECS tasks are too small to run dbt (to materialize the asset). I need to keep the default small, but I want to set higher values for all the dbt assets.
What did you expect to happen?
The asset job should be launched successfully in an ECS task with the specific resources. Those are valid tags that can be applied to jobs or assets defined in the "Pythonic" way (but components are recommended now, so they should be supported too). And if I manually add the tags via the Launchpad, they work.
How to reproduce?
here is my defs.yaml:
type: dagster_dbt.DbtProjectComponent
attributes:
project: '{{ env.CLICKHOUSE_DBT_PROJECT_DIR or (project_root + "/../clickhouse/dbt") }}'
post_processing:
assets:
- attributes:
tags:
ecs/task_overrides: '{"cpu": "1024", "memory": "2048"}'
ecs/cpu: "512"
ecs/memory: "1536"
Dagster version
1.11.11
Deployment type
Other Docker-based deployment
Deployment details
self hosted open source on AWS ECS
Additional information
No response
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.