Easier display of dashboard

[stsievert]

Currently, I have to port-forward twice: once for Jupyter and once for the Dask Dashboard. It'd be nice to only require one of those:

ssh -L 8888:localhost:8888 8787:localhost:8787 stsievert@submit3.chtc.wisc.edu

Luckily, Dask has a Jupyterlab extension: https://github.com/dask/dask-labextension. Would it be possible to integrate this into Dask-CHTC? Then, I'd only have to forward one port I think:

ssh -L 8888:localhost:8888 stsievert@submit3.chtc.wisc.edu

[JoshKarpel]

Unfortunately, as far as I know, this isn't possible. In my testing, it appears that the extension is just pointing at the normal dashboard web address (it doesn't have privileged access to the dashboard). Since the extension is running in your browser, it needs to be able to see the dashboard address. Would love to be proven wrong though!

[stsievert]

I think this comment might be relevant: dask/dask-labextension#41 (comment). In short, there are good security reasons to avoid dashboards. I think this is cluster handler he's talking about: https://github.com/dask/dask-labextension/blob/46cbdc102412b98ac8d67f9a8abed2ca3b332a8d/dask_labextension/clusterhandler.py#L14

[JoshKarpel]

Huh. He seems to be implying that if you start your Dask cluster through the cluster manager then the dashboard will be available via a backdoor. I haven't tried that out, but I do know that you can spawn a CHTCCluster using the labextension if you set the up the labextension config file correctly (#19 and #15). Would you mind trying that out and seeing if you can access the dashboard for a cluster created via that method without forwarding extra ports for it?

[stsievert]

Would you mind trying that out and seeing if you can access the dashboard for a cluster created via that method without forwarding extra ports for it?

I might be able to try it out with some documentation; I'm not sure how to setup the labextension config correctly.

[JoshKarpel]

Gotcha; a minimal ~/.config/dask/labextension.yaml would be

labextension:
  factory:
    module: 'dask_chtc'
    class: 'CHTCCluster'
    args: []
    kwargs: {}
  default:
    workers: null
    adapt: null

[JoshKarpel]

Well, I gave this a shot and found #33 and #34, but unfortunately it looks like there's no magic for the original issue: I wasn't able to get the lab extension to see the dashboard, even with a cluster/client set up by it. I'll leave this open for tracking further developments.