See Dask
Installation is easy via a JupyterLab extension: https://github.com/dask/dask-labextension
Security
One worry is that Dask currently communicates over TCP sockets, not unix sockets. So different users can interact with each others' Dask clusters. To remedy this we have to set the default LocalCluster config to use tls cert authentication between node and workers.
Below is an explanation of the issue and how to remedy it using tls client certification config. Note that there may be a performance penalty (slight) imposed by making the connection between client and LocalCluster use tls client certs. Ideally, the user in the JupyterLab frontend should be able to turn this off/on as desired.
Copilot explanation of the security issue and how to remedy
When you start a LocalCluster using the Jupyter Dask extension, it does not implement strict security or encryption by default. It relies primarily on network binding rather than user authentication, meaning it binds to 127.0.0.1 (localhost).
Because it binds to localhost, users from the outside network cannot access it, but other local users logged into that same VM can still connect to your cluster if they scan for and discover your open ports.
To properly isolate your cluster from other users on the shared machine, you must configure the extension to enforce automated TLS security.
How to Force TLS Security in the Jupyter Extension
Dask has a built-in feature where passing security=True to a LocalCluster tells it to automatically generate temporary, self-signed TLS credentials in memory. This prevents other local users from hijacking the connection because they lack your specific in-memory certificates.
You can configure the Jupyter Dask Extension to automatically use this security layer every time you click the + NEW button.
Step 1: Update your Dask Configuration
Edit your global Dask configuration file at ~/.config/dask/dask.yaml (create it if it does not exist) and add security: true to the LocalCluster keyword arguments under the labextension factory settings: [2, 9]
labextension:
factory:
module: 'dask.distributed'
class: 'LocalCluster'
args: []
kwargs:
# This forces the extension to generate secure TLS certs automatically
security: true
Step 2: Connecting via your Notebook
When you click + NEW in the Jupyter sidebar, Dask will spin up the cluster securely using the tls:// protocol.
When you click the <> button to inject the client connection code into your notebook, it will look like this:
from dask.distributed import Client
The extension automatically handles passing the matching TLS credentials into your notebook session behind the scenes.client = Client("tls://127.0.0.1:XXXXX")
By enforcing security: true, any other user on the VM trying to connect to your cluster's port (tls://127.0.0.1:XXXXX) will be blocked by the TLS handshake because they do not have your ephemeral encryption keys.
See Dask
Installation is easy via a JupyterLab extension: https://github.com/dask/dask-labextension
Security
One worry is that Dask currently communicates over TCP sockets, not unix sockets. So different users can interact with each others' Dask clusters. To remedy this we have to set the default LocalCluster config to use tls cert authentication between node and workers.
Below is an explanation of the issue and how to remedy it using tls client certification config. Note that there may be a performance penalty (slight) imposed by making the connection between client and LocalCluster use tls client certs. Ideally, the user in the JupyterLab frontend should be able to turn this off/on as desired.
Copilot explanation of the security issue and how to remedy
When you start a LocalCluster using the Jupyter Dask extension, it does not implement strict security or encryption by default. It relies primarily on network binding rather than user authentication, meaning it binds to 127.0.0.1 (localhost).
Because it binds to localhost, users from the outside network cannot access it, but other local users logged into that same VM can still connect to your cluster if they scan for and discover your open ports.
To properly isolate your cluster from other users on the shared machine, you must configure the extension to enforce automated TLS security.
How to Force TLS Security in the Jupyter Extension
Dask has a built-in feature where passing security=True to a LocalCluster tells it to automatically generate temporary, self-signed TLS credentials in memory. This prevents other local users from hijacking the connection because they lack your specific in-memory certificates.
You can configure the Jupyter Dask Extension to automatically use this security layer every time you click the + NEW button.
Step 1: Update your Dask Configuration
Edit your global Dask configuration file at ~/.config/dask/dask.yaml (create it if it does not exist) and add security: true to the LocalCluster keyword arguments under the labextension factory settings: [2, 9]
labextension:
factory:
module: 'dask.distributed'
class: 'LocalCluster'
args: []
kwargs:
# This forces the extension to generate secure TLS certs automatically
security: true
Step 2: Connecting via your Notebook
When you click + NEW in the Jupyter sidebar, Dask will spin up the cluster securely using the tls:// protocol.
When you click the <> button to inject the client connection code into your notebook, it will look like this:
The extension automatically handles passing the matching TLS credentials into your notebook session behind the scenes.client = Client("tls://127.0.0.1:XXXXX")
By enforcing security: true, any other user on the VM trying to connect to your cluster's port (tls://127.0.0.1:XXXXX) will be blocked by the TLS handshake because they do not have your ephemeral encryption keys.