JSD v0.2.x is incompatible with JupyterLab 4.6

[dlqqq]

Summary

jupyter-server-documents (JSD) v0.3.0b0 does not work on JupyterLab v4.6 (tested with jupyterlab==4.6.1). This is a downstream consequence of #262: JupyterLab 4.6 bundles @jupyter/ydoc v4, but JSD's prebuilt frontend — and its @jupyter/collaboration / @jupyter/docprovider dependencies — are built against @jupyter/ydoc v3 (>=2.1.3 <3.0.0 || >=3.0.0 <4.0.0).

Until #262 lands (upgrading JSD to @jupyter/ydoc v4), JSD should be considered incompatible with JupyterLab >= 4.6, and I'd suggest documenting/pinning jupyterlab <4.6 for the v0.3.0 line.

Related

• Blocked on / duplicates the root cause tracked in Upgrade Jupyter YDoc v4 #262 ("Upgrade Jupyter YDoc v4").
• Note: Jupyter Collaboration v4 almost certainly has the same problem on JupyterLab 4.6 — its v5 release (which moves to @jupyter/ydoc v4) is still pre-release (5.0.0 is not out yet), so the currently-released jupyter-collaboration line is also pinned to ydoc v3.

Reproduction

pip install "jupyterlab==4.6.1" jupyter-server-documents
jupyter lab

What the user sees

jupyter labextension list reports the extensions as incompatible:

"@jupyter-ai-contrib/server-documents@0.3.0-b0" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab              Extension               Package
>=4.0.0-b1 <5.0.0       >=2.1.3 <3.0.0 || >=3.0.0 <4.0.0    @jupyter/ydoc
"@jupyter/collaboration-extension@4.4.1" is not compatible ...   (same @jupyter/ydoc conflict)
"@jupyter/docprovider-extension@4.4.1" is not compatible ...     (same @jupyter/ydoc conflict)

In the browser, the console is full of module-federation shared-singleton rejections, the decisive one being:

Unsatisfied version 4.0.0 from CORE_FEDERATION of shared singleton module @jupyter/ydoc (required ^3.0.0-a3)

(accompanied by a cascade of @jupyterlab/* singleton mismatches, since the JSD-built assets expect the 4.5-era @jupyterlab/* alpha ranges).

Why exactly it fails

@jupyter/ydoc is declared a shared singleton in the module-federation graph. When both JupyterLab (v4) and JSD (v3) request it, webpack federation can only hand out one instance for the whole app. JupyterLab wins as the host, so JSD's v3-built code receives the v4 @jupyter/ydoc — whose YNotebook / YFile / awareness classes are not API-compatible with what JSD compiled against.

The visible result is not a hard white-screen crash — the shell renders and plugins report as "activated" — but documents never populate: opening a notebook shows an empty editor with the status bar stuck at Cell 0/0 and the kernel indicator stuck at Initializing (see screenshot). The shared model handed to JSD is the wrong ydoc version, so the notebook's cells/outputs are never wired up and the document is effectively unusable.

Environment

• jupyterlab==4.6.1 (bundles @jupyter/ydoc 4.0.0)
• jupyter-server-documents==0.3.0b0
• jupyter-collaboration-ui==2.4.1, jupyter-docprovider==2.4.1 (both ydoc v3)

Suggested resolution

• Land Upgrade Jupyter YDoc v4 #262 (upgrade JSD frontend to @jupyter/ydoc v4 + the JupyterLab 4.6 @jupyterlab/* ranges), and adopt jupyter-collaboration v5 once it's released.
• In the meantime, add a jupyterlab <4.6 ceiling (or a documented note) so users don't silently land on a broken combination.

[dlqqq]

screenshot from an ad-hoc E2E test I ran:

[dlqqq]

Weird, even though it's causing E2E test failures and a user reported an issue at the Jupyter AI community call, I can't reproduce anything locally. I see the console warnings but things still seem to be working OK with JupyterLab v4.6.0. Maybe this isn't worth blocking JSD v0.3.0 over?

[krassowski]

@jupyter/ydoc is declared a shared singleton in the module-federation graph

That's weird. I do not see a direct declaration asking for it to be a singleton anywhere. Is it transitive through

jupyter-server-documents/package.json

Lines 142 to 149 in 450d05e

	"sharedPackages": {
	"@jupyter/chat": {
	"singleton": true
	},
	"jupyterlab-chat": {
	"singleton": true
	}
	}

or some other magic?

Would bumping the requirement here work?

jupyter-server-documents/package.json

Line 65 in 450d05e

"@jupyter/ydoc": "^2.1.3 || ^3.0.0",

so JSD's v3-built code receives the v4 @jupyter/ydoc — whose YNotebook / YFile / awareness classes are not API-compatible with what JSD compiled against

The API is nearly 99% the same the only difference is the dirty flag behaviour.

[krassowski]

a user reported an issue at the Jupyter AI community call,

They reported an issue building a custom extension depending on both, not on runtime, right?

[krassowski]

Final piece of context: JupyterLab 4.6.0 works fine with jupyter-collaboration built against @jupyter/ydoc v3. The goal (as always) was to make the minor release easy to upgrade to, so I am surprised by the issue here - I would like to understand what it is and what we may need to do differently next time.

[dlqqq]

Discussed this on the call - I wasn't seeing any runtime issues, I think the errors only happened in CI environments for some reason. I did see a memory leak regression introduced by the new version of pycrdt required by Jupyter YDoc v4 (documented in #270), but found a workaround in #264 and that is now merged. This should be fixed in JSD v0.3.0. 👍

@krassowski Thanks for the help investigating this. Agree the Jupyter YDoc v3 => v4 change was not the main issue, it was really the new release of pycrdt that it brought along. I'm pretty sure JupyterLab v4.6 does somehow require Jupyter YDoc v4 -- I got multiple reports of that and did manage to reproduce it locally, I'll have to check again to make sure I'm not making things up.