Skip to content

load_assets_from_modules and include_specs=True prevents horizontal importing #33282

@easontm

Description

@easontm

What's the issue?

My asset list has a mix of Assets and AssetSpecs (my upstream source data), which is why I am including specs.

In my setup, I have dag/file1.py and dag/file2.py. I import an asset from file1 into file2 so I can avoid typing out AssetKeys manually when setting deps. With include_specs=False, this works fine. With it true, I get an error about duplicate assets.

For now I can just not include AssetSpecs using the loader function and send them to my Definitions manually, but the loader function is nice because it lets me ensure asset group/prefix across the whole code deployment.

What did you expect to happen?

I can include specs and import assets across my files for dependency references.

How to reproduce?

# dag/__init__.py
autoloaded = dg.load_assets_from_modules(
    modules[file1, file2],
    include_specs=True,
)

defs = dg.Definitions(assets=autoloaded)
# dag/file1.py
@dg.asset
def asset1():
    ...

# dag/file2.py
from dag.file1 import asset1
@dg.asset
def asset2(asset1):
    ...

Dagster version

1.12.3

Deployment type

Dagster Helm chart

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions