jupytext and jupyter lite ?

[parmentelat]

Hi; I am currently experimenting with jupyter lite, and found that jupytext apparently does not work in this context at the moment

I am curious to hear of others who would have this interest too, to share experiences and possibly best practices

[mwouts]

Hi @parmentelat , thanks for opening this thread. I see are at least two difficulties for providing the current Jupytext in Jupyter Lite
a) The context manager for Jupyter Lite seems to be very different from the Jupyter Lab one (e.g. it's coded in Javascript, while Jupytext's contents manager is written in Python)
b) I don't know how to distribute a Python package for pyodide. I tried

import pyodide_js
await pyodide_js.loadPackage('jupytext')

but it failed with the following error: JsException: Error: No known package with name 'jupytext'

Possibly it would make sense to code another converter from scratch in TypeScript, but I am afraid I don't have the skills nor the time to do so. @jtpio do you have any thoughts on the subject?

[parmentelat]

Hey @mwouts - hope you're doing fine

well, as far as skills I am unfortunately not the right person either :)

I was just raising this because I thought I might not be the only jupytext addict to start playing with jupyterlite, given that it is becoming a more and more viable and appealing option...

[nthiery]

Same here: can't wait to have jupytext in jupyterlite. IIRC from last time I tried, it's not far from working:
using the conda-based jupyterlite xeus, I could just pull jupytext
by listing it in the pip dependencies in the environment. And then I could open a markdown notebook
with Open With.

[nthiery]

Cross referencing discussions:

• Support for Jupytext jupyterlite/jupyterlite#731
• serving of text-based notebooks using jupytext jupyterlite/jupyterlite#1301

[ValMystletainn]

Same here: can't wait to have jupytext in jupyterlite. IIRC from last time I tried, it's not far from working: using the conda-based jupyterlite xeus, I could just pull jupytext by listing it in the pip dependencies in the environment. And then I could open a markdown notebook with Open With.

I'm tring to use jupytext in jupyter lite too. But I have failed to do so. So I want to check the opearation with you. What I have doen is

1. create an environment.yaml and put the xeus-python, jupytext into it's dependency
2. do jupyter lite build --XeusAddon.environment_file=./environment.yaml to build the jupyterlite bundle with xeus and jupytext init
3. run jupyter lite serve to push the jupyter lite into localhost server.
4. visit the corresponding address. I have got the jupytext customized pages, but when I right click the .md file or .py file and open it with jupytext notebook, I still got the json error

[mwouts]

@krassowski @martinRenou would you have a suggestion for where to look for? Is there anything like the concept of a "contents manager" for jupyter lite? If so, can it run Python code?

[martinRenou]

It does not have a concept of server extensions that can run Python.

So you'd need to provide a JS implementation of the server extension of Jupytext. I wonder if the new ContentProvider API of JupyterLab could help jupytext hook into some custom logic to handle files? Could Jupytext become fully in-browser with this?

[mwouts]

It does not have a concept of server extensions that can run Python.

I see - but the notebooks themselves can run Python ? Is that a completely different part of the codebase ?

So you'd need to provide a JS implementation of the server extension of Jupytext. I wonder if the new ContentProvider API of JupyterLab could help jupytext hook into some custom logic to handle files? Could Jupytext become fully in-browser with this?

Yep we scratched the surface of the ContentProvider API earlier with @krassowski . I did consider the idea to re-implement Jupytext in TypeScript but I am not familiar enough with the language. In addition many formats (like MyST Markdown) are actually implemented outside of Jupytext (but still in Python) so I think what could potentially unlock this issue is if I had an example that shows how to pre-process a file (with Python) in jupyterlite before it's displayed?

[martinRenou]

I see - but the notebooks themselves can run Python ? Is that a completely different part of the codebase ?

Yes, those are Jupyter kernels which is different from the server in the Jupyter architecture.

example that shows how to pre-process a file (with Python) in jupyterlite before it's displayed?

You wouldn't preprocess it in Python ideally.

A custom content provider could look like this:

class JupytextContentProvider implements IContentProvider {
  constructor(options) {
    this._currentDrive = options.currentDrive;
  }

  /*
   * "get" the file from the server, then preprocess it
   **/
  async get(
    localPath: string,
    options?: Contents.IFetchOptions
  ): Promise<Contents.IModel> {
    let filecontent = this._currentDrive.get(localPath, {
      ...options,
      content: true,
      contentProviderId: undefined
    });

    // DO YOUR PREPROCESSING HERE IF NEEDED!
    // if (localPath.endswith('.md')) { filecontent = markdownToNotebook(filecontent) }

    return filecontent;
  }

  /*
   * reprocess the file into valid markdown, then "save" the file on the server
   **/
  async save(
    localPath: string,
    options: Partial<Contents.IModel> & Contents.IContentProvisionOptions = {}
  ): Promise<Contents.IModel> {
    // DO YOUR OPPOSITE OF PREPROCESSING HERE IF NEEDED!
    // if (localPath.endswith('.md')) { options = notebookToMarkdown(options) }

    this._currentDrive.save(localPath, {
      ...options,
      contentProviderId: undefined
    });
  }

  private _currentDrive: Contents.IDrive;
}

const defaultDrive = (app.serviceManager.contents as ContentsManager)
  .defaultDrive;
const registry = defaultDrive.contentProviderRegistry;

const jupytextContentProvider = new JupytextContentProvider({
  currentDrive: defaultDrive
});
registry.register('jupytext', jupytextContentProvider);

You would then need to register the markdown filetype (overwrite the existing one?) so that it uses the "jupytext" content provider.

like MyST Markdown

MyST-JS is a thing apparently, so you could use it.