closing a cluster with adaptive turned on logs asyncio error messages

[mariusvniekerk]

What happened:

Running a simple adapt LocalCluster logged the following errors on termination of the cluster

DEBUG:asyncio:Using selector: KqueueSelector
DEBUG:asyncio:Using selector: KqueueSelector
DEBUG:asyncio:Using selector: KqueueSelector
ERROR:asyncio:Task was destroyed but it is pending!
task: <Task pending name='Task-75' coro=<AdaptiveCore.adapt() done, defined at /Users/mvanniekerk/miniconda3/envs/blockscorepy38/lib/python3.8/site-packages/distributed/deploy/adaptive_core.py:179> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fe3eef86dc0>()]> cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/mvanniekerk/miniconda3/envs/blockscorepy38/lib/python3.8/site-packages/tornado/ioloop.py:688]>

What you expected to happen:

This not to happen?

Minimal Complete Verifiable Example:

import distributed
import logging
logging.basicConfig(level=logging.DEBUG)

n_workers = 4


def run():
    with distributed.LocalCluster(n_workers=1, threads_per_worker=1) as cluster:
        adapt = cluster.adapt(minimum=1, maximum=n_workers)
        try:
            with distributed.Client(cluster) as client:
                pass
        finally:
            pass
            # adapt.stop()


if __name__ == '__main__':
    run()

Anything else we need to know?:

If adapt.stop() is called this doesn't happen

Environment:

• Dask version: 2020.12.0
• Python version: 3.8
• Operating System: osx-64
• Install method (conda, pip, source): conda

[jacobtomlinson]

I'm curious how this came about. I would assume that LocalCluster doesn't support adapt because it has no way to create new resources.

Should we stub out the method so that this can't be run in the first place?

[mariusvniekerk]

LocalCluster very much does support adapt as shown by
https://gist.github.com/mariusvniekerk/edeefd93b17a7ae01472e1818512ac55