SciPy 2022 Sprint Planning

[jacobtomlinson]

Dask Control is a CLI application for managing the lifecycle of your dask clusters. In #19 we merged an experimental new text based UI which gives you a nice interface for managing your clusters and also a way to view the dask dashboard via the CLI.

For the SciPy 2022 sprint I'd love folks to help me work on this and make it better. To do so you'll need to learn about rich, textual and dig into the dask scheduler state machine. Those are just cool things to play with regardless, so if you spend the whole weekend going off on a rich or textual tangent it will be time well spent.

There are a bunch of things that could be done in dask-ctl:

• Build plots from the web dashboard for the TUI (probably the best task for folks to get started with quickly)
• Replace Dask logo with new one
• Update textual to more recent version
• Implement graph selector where you can view any plot in full screen
• Write tests
• Write documentation
• Anything else that would make it cool

I initially had a go at recreating the the dask dashboard status page, but the implementations have gone stale and currently don't work so if you view the dashboard in the tui you'll just see placeholders right now.

I'd really love to populate these spots and beyond. So let's talk through the structure of the project and some setup info.

Setup

You'll need dask and dask-ctl installed in developer mode, here is how I would get that set up but feel free to do it however you like.

$ conda create -n dask-tui -c conda-forge python=3.10 numpy dask distributed ipython -y

$ conda activate dask-tui

$ git clone https://github.com/dask-contrib/dask-ctl.git

$ cd dask-ctl

$ pip install -e .

$ pip install -r requirements.txt

Then you can try the tui out. It will help if you start a dask scheduler first so you can see something in the list.

$ dask-scheduler  # Run this in a different window as it will be long running

$ daskctl ui

Then if you hit enter on your cluster in the list it'll drop you into the dashboard view, and escape will take you back to the list.

Widgets

The text UI is built with textual which is a widget-based system. If you explore dask_ctl.tui.widgets you'll find the code for all the widgets that have been built so far. Many do not work due to upstream changes in textual and dask since they were experimented with at the start of the year.

You'll also find a graphs submodule that contains shared code for drawing graphs in the terminal.

dask_ctl/tui
├── __init__.py
├── events.py
├── graphs
│   ├── __init__.py
│   ├── bar.py
│   ├── taskstream.py
│   └── tests
│       └── test_utils.py
├── tui.py
└── widgets
    ├── __init__.py
    ├── cluster_info.py
    ├── cluster_table.py
    ├── command_reference.py
    ├── info.py
    ├── key_bindings.py
    ├── logo.py
    ├── nbytes.py
    ├── processing.py
    ├── progress.py
    ├── prompt.py
    └── taskstream.py

Inspired by rich many of these files can be run directly with python and they will show a small example or demo. They work by having a small if __name__ == "__main__": that created a textual app with just that one widget.

$ python dask_ctl/tui/widgets/logo.py

Many examples grab a pointer to the dask-scheduler you have running on port 8786 so that you can build a graph that shows some specific information. So a good place to start is to copy an existing widget and replace the class with your own widget. You can just run it with python for testing and then when it's done we can wire it up inside the main UI.