Notes on running docker on jupyter lab

[adswa]

a ping for @mih. Here are my explorations on running docker on the littlest jupyterhub on AWS.

I tried it with code from https://handbook.datalad.org/en/latest/code_from_chapters/neurohackademy.html.

Because docker buried its non-Docker-desktop related docs quite a bit, i first followed a slightly outdated stack from https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04. It worked, but it uses deprecated stuff. I later found this more modern documentation which sounds like a better alternative.

sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo -E apt install docker-ce

add users to the docker group

sudo usermod -aG docker jupyter-turmeric

This still resulted in the following error for a non-privileged user account:

❯ datalad containers-add nilearn \
      --url dhub://djarecka/nilearn:yale
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/create?fromImage=djarecka%2Fnilearn&tag=yale": dial unix /var/run/docker.sock: connect: permission denied
CommandError: 'docker pull djarecka/nilearn:yale' failed with exitcode 1

so I chmod'ed the socket

sudo chmod 666 /var/run/docker.sock

When executing a script inside of the container (following the docker-related code snippets from https://handbook.datalad.org/en/latest/code_from_chapters/neurohackademy.html), I saw a warning I haven't seen elsewhere or with a singularity container

whoami: cannot find name for user ID 1002
Matplotlib created a temporary config/cache directory at /tmp/matplotlib-lbgcqndp because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.

However, everything worked, containers-add, containers-list, containers-run and rerun.

[adswa]

Also note that a push to external hosting can be slower depending on the service. OSF was very slow compared to a singularity image because the Docker image is saved as several layers instead of a single file.