Running third-party packages like numpy and pandas

[psads-git]

I find pyrunner an extremely useful typst library. However, it has an important limitation: It cannot run third-party libraries. Package pyodide may lift this so constricting limitation (pyodide is a Python distribution for the browser and Node.js based on WebAssembly):

https://app.scribbler.live/?jsnb=github:gopi-suvanam/scribbler-examples/Python-in-Browser.jsnb

With pyodide one can run numpy, pandas, etc.

So, I would suggest to enhance pyrunner in that direction.

[peng1999]

We can draw inspiration from Pyodide, but it offers limited help here. The key point is that Pyodide runs on the Emscripten target. The Emscripten runtime provides rich APIs, including FS APIs, which are essential for the CPython import process, while the Typst plugin runtime does not.

To integrate CPython into the Typst plugin system, we must either create a dummy file system and provide API to CPython or make a custom Python module loader to bypass filesystem reads. Even if we overcome this, we'll still need to address packaging wasm versions of C extensions and Typst's poor wasm performance.

That said, none of the above is impossible. However, I'm concerned if the gain is worth the effort. Anyways you can run Python code outside the Typst file and include the output in it. Can you provide a use case where pyrunner is better than alternatives if package import is implemented?

[psads-git]

Thank you, @peng1999, for your response.

I understand that we can run Python code outside of Typst and then import the results into Typst. However, this approach can become cumbersome as the number of code snippets increases and changes in the code snippets need to be performed. This is why I believe a library like Pyrunner could help alleviate these issues.

Quarto, for example, handles this process in a very seamless way, as demonstrated in the attached video. However, it's important to note that Quarto does not support automatic preview by design.

The scenario of creating lecture notes with integrated Python code snippets is one of the use cases.

quarto_example.mp4

[peng1999]

I don't think literate programming will be a valid use case of pyrunner. There are some unsolvable problems:

• Performance: The Python interpreter running in Typst is magnitude slower than that running on normal OS.
• Interactivity: All calculation is controlled by Typst and we have no way to do actions like Run Cell or Run Above. This will lead to even worse performance.

I think we should wait for Typst to have a native WASM interpreter in the browser before reconsidering the feasibility of literate programming within it.

[psads-git]

Thank you for your response and for explaining the challenges with adding literate programming to pyrruner. I understand the current performance and interactivity limitations within Typst, and I agree that waiting for a native WASM interpreter might be the best course of action.

I appreciate your work on pyrruner and look forward to future developments.

[RaulDurand]

For reference, there is the Jlyfish package to run Julia code. It uses a watch command in Julia and store the results in a Json file. It may be used as inspiration.