Logging configuration using environment variables

[dbalabka]

Describe the issue:
The logging configuration can not be set using environment variables. However, it is the easiest way to pass some configuration to the cluster's workers. Specifically, I would like to update the following settings:

dask.config.get("distributed.logging")

{'distributed': 'debug', 'distributed.client': 'debug', 'distributed.scheduler': 'debug', 'distributed.shuffle': 'debug'}

Minimal Complete Verifiable Example:
The following code throws an error

import os

os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__CLIENT'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__SCHEDULER'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__SHUFFLE'] = 'info'

import dask
import dask.distributed

Trace:

Traceback (most recent call last):
  File ".../test.py", line 8, in <module>
    import dask.distributed
  File ".../.venv/lib/python3.10/site-packages/dask/__init__.py", line 3, in <module>
    from dask import config, datasets
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 802, in <module>
    refresh()
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 522, in refresh
    update(config, collect(**kwargs))
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 485, in collect
    configs.append(collect_env(env=env))
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 270, in collect_env
    set(d, config=result)
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 391, in __init__
    self._assign(key.split("."), value, config)
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
    self._assign(keys[1:], value, d[key], path, record=record)
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
    self._assign(keys[1:], value, d[key], path, record=record)
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
    self._assign(keys[1:], value, d[key], path, record=record)
  File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 450, in _assign
    d[key] = value
TypeError: 'str' object does not support item assignment

Environment:

• Dask version: 2024.1.0
• Python version: 3.10
• Operating System: Win11, WSL
• Install method (conda, pip, source): poetry

[dbalabka]

Seems that the issues already known:
#6124 (comment)

However, I believe it can be fixed.