Skip to content

Can't upload logs from task runner setup #17776

Open
@bnaul

Description

@bnaul

Describe the current behavior

Currently the task runner gets initialized before we enter the flow run context. This means that if we're for example creating a Dask cluster, any logs about how the cluster was created (or fails to get created) do not get associated with the flow run and can't be viewed in the UI.

Describe the proposed behavior

It seems like a little tweak to the ordering here would be sufficient: as long as we call task_runner.__enter__ after FlowRunContext.__enter__, any logs/events would propagated as expected which would make debugging Dask (and I assume Ray) cluster issues much simpler.

Example Use

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAn improvement of an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions