Skip to content

Add preload and preload_argv arguments to Client.__init__ #5819

Open
@bryanwweber

Description

@bryanwweber

PR #5773 adds the ability for the Client to run preload code, similar to the Worker and Scheduler. However, @jrbourbeau pointed out that users may be passing preload and preload_argv to the Client.__init__ with the intention of passing them to the worker configuration of LocalCluster.

As a temporary measure, #5773 only adds Client preload code that is specified in the Dask YAML configuration file. Since a similar problem with keyword arguments passthrough will happen if/when plugins are added for the Client, @jrbourbeau proposed deferring adding the preload and preload_argv keywords. When those are added, there will need to be a deprecation cycle to change how users pass kwargs to the worker configuration through LocalCluster. This deprecation cycle would also remove the similar problem that will occur when/if plugins are added.

One possible solution to this is to modify the signature of LocalCluster.__init__ so that the **worker_kwargs that are collected at the end of the constructor become a keyword argument of their own, similar to the existing scheduler_kwargs. See

scheduler_kwargs=None,
scheduler_sync_interval=1,
**worker_kwargs,

This should not affect the n_workers and threads_per_worker kwargs, which would still be passed through unmodified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprove existing functionality or make things work bettergood second issueClearly described, educational, but less trivial than "good first issue".

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions