Jupyter notebooks not working in VS Code; updating Python extension build results in unlaunchable image

[nickeubank]

Hi Coursera!

I'm working on developing a course with some colleagues at Duke, and we've been trying to get Jupyter notebooks and interactive window up and running for our VS Code lab images, but to no avail. With the default VS Code image, notebooks won't run. I can get a notebook file recognized and the interface shows up, but I can't get cells to run:

I've installed jupyter in both miniconda and the /usr/python installs, but to no avail.

And if I try and use Jupyter extension commands from the Command Pallette, I get this kind of stuff:

I spent a while working with some very generous folks at Coder trying to figure out it, and they had me uninstall the Python and jupyter extensions in the default image and manually load .vsix files, which worked (Python: ms-python.python-2021.10.1365161279.vsix, and Jupyter: ms-toolsai.jupyter-2021.8.12.vsix). But when I tried to just bump the Python build in your Dockerfile (hoping it was a build issue), the result was an image that I couldn't load—I just got a pulsing circle over and over when I tried to load the image.

So... Any suggestions?

@kylebradbury

[echuber2]

Hello from the University of Illinois at Urbana-Champaign. First, I'd note that you might be better off using the Jupyter Notebook lab instead of using the (experimental) Jupyter features within VS Code.

It looks like the vsix was not successfully installed to begin with in the lab image; there are missing modules in the install directory (as mentioned in an error in the browser dev console). The versions of everything in the lab are old now, but until Coursera updates everything, here is a fix:

When you configure the lab based on Coursera's provided base/vscode-v2.0-custom-grader-v2.0 image, you need to add this in the image configurator where it says "Configure Image With Docker Commands (Optional)":

RUN cd /tmp && wget https://open-vsx.org/api/ms-toolsai/jupyter/2021.8.12/file/ms-toolsai.jupyter-2021.8.12.vsix && code-server --force --install-extension ./ms-toolsai.jupyter-2021.8.12.vsix
RUN conda install --quiet --yes jupyter

Then rebuild the image and make a fresh lab attached to it for a clean start in the instructor lab initialization.

This is redownloading and reinstalling a copy of the same version of the necessary vsix from the third-party open-vsx.org mirror. (You'll have to trust it. It's run by the people behind the Eclipse project. By the way, Coursera still has a version of code-server prior to v4 installed, but versions of code-server from v4 onward also use open-vsx instead of the Microsoft extension repository. The official Microsoft repository is license restricted to official VS Code builds only. More info here. This is something that Coursera will have to deal with when they upgrade everything as not all extensions will still be available in the same way as before. We had to deal with it on our university's PrairieLearn system too.)

I also added the conda install line for the jupyter package, as otherwise it would try to download that on-demand in the user session later. If you're going with conda you'll have to install whatever other packages your users will need too. Since multiple potential Python kernels will exist in the lab image, users will be prompted to pick the right one later, and you'll need to guide students in this.

I recommend not trying to use the command palette option that's supposed to create a new blank notebook. It seems buggy if you try to use it more than once. Instead, manually create a blank .ipynb file (using "touch test.ipynb" in the terminal, etc.) and then just open it manually. You can give students a starter file.

Here's an illustration of this working in the instructor lab. I also tested in a published learner lab and it still worked there too. Remember that the learner will need to select the correct kernel (that matches the one for which you installed all your libraries).

[echuber2]

One other note: You may want to continue using conda for the packages and kernel, as the lab does already have the right paths set up for it, but not for pip-installed stuff (apparently). And it's easy to just tell students "pick the conda one" from the kernel dropdown.

[nickeubank]

AMAZING! Works perfect, thanks so much @echuber2 !

[mvhudson-duke]

@profmikegreene this is the issue I was talking about, but hopefully this sheds some light on some of the stuff we've been working through!

[echuber2]

It may be worth reopening the issue until it's fixed in Coursera's official image.