feat(markdown): add support for math rendering with typst and katex

[cestef]

IMPORTANT: Please do not create a Pull Request adding a new feature without discussing it first.

The place to discuss new features is the forum: https://zola.discourse.group/
If you want to add a new feature, please open a thread there first in the feature requests section.

Sanity check:

• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Add optional math support for markdown #2708 stalled

Code changes

(Delete or ignore this section for documentation changes)

• Are you doing the PR on the next branch?

If the change is a new feature or adding to/changing an existing one:

• Have you created/updated the relevant documentation page(s)?

[cestef]

I don't really know how to get rid of these Cargo.lock updates, but anyways

[cestef]

This PR adds math rendering on the "server-side" with typst, while keeping the katex option open for latex.

A GenericCache struct has also been added for integrations like these to store the compilation results to save time on subsequent builds. Ideally, each integration would have its cache in the the global <Site>.caches attribute. Cache is stored as binary-encoded data at .cache/<CACHE_NAME>. Typst remote packages are also downloaded to this path.

The output SVGs generated by typst can also be minified via the svgo CLI. This is kind of a hacky implementation because it relies on the CLI but it does the job.

Newly added configuration parameters:

math = "typst"                       # Defaults to "none", one of "typst", "katex"
math_svgo = true                     # Optimize the SVGs generated by the math renderer
math_svgo_config = "svgo.config.mjs" # Optional, Path to the svgo config
math_css = "styles/typst-embed.css"  # path to the CSS file to inject inside the SVGs generated 

CI is failing because [email protected] (used by typst and image) and [email protected] (used for caching keys) require rustc >= 1.81, should we upgrade rustc or downgrade these dependencies ?

The size of the binary also drastically increases because of typst's ecosystem (36.4 MiB -> 62.0 MiB)

[claytonwramsey]

This is great work! One possible place to improve: the current implementation outputs SVG for equations. It might be better to use MathML to make sites smaller and more accessible.

[cestef]

Thanks a lot :D Typst doesn't yet support HTML or MathML output (see typst/typst#5512)

The basic katex implementation however already outputs HTML I think. I don't use LaTeX that much so if anyone is up to help with this, I'd be glad.

[claytonwramsey]

I think it might be possible to write a backend visitor to output MathML (cf https://github.com/wcshds/typst-math-to-mathml-converter). This would be a lot easier than what the Typst team is building, since we don't have to handle everything else, but the downside is that we're then implementing a possibly buggy different spec from what Typst does.

As a stopgap for accessibility, we might also just include the math source as alt text for the generated SVGs.

[cestef]

This indeed sounds a bit risky, including the original source in the alt should be enough for now. I will tinker a bit with the library you linked to try out mathml rendering, I also stumbled accross this one:

https://github.com/xushengfeng/xmmath

[cestef]

I have tried out a bit the library you linked, here's the result I get:

I am using katex's font for the rendered mathml. There are still a lot of inconsitencies (the spaces between $a$ and $x$, the spacing feeling weird), but overall it seems like a viable solution if we were to tweak things a bit.

For reference, here's the result with SVG rendering:

[soqb]

this looks great! i've left a few comments but i've been toying locally for a few days and i'm very happy.

+1 for svg+mathml too. it's always what i gravitate to and it always works well. one day soon mathml will be widespread and complete enough that it will become a viable primary target, but for now, svg rendering is the only consistent option.

on that note, did you ever consider mathjax? the latex support is a little better than katex these days.

[soqb]

i'm not totally sure whether it matters at the kind of scale/concurrency zola operates on, but you might find papaya a better fit for a concurrent hash map. this (biased) blogpost by the author totally convinced me.

[soqb]

performance wise, though it's somewhat (but not totally) moot, you might want to clone fence.language beforehand so you don't have to pay the larger cost of cloning inside CodeBlock::new.

[soqb]

is there a reason katex isn't supported here too?

[cestef]

I don't use latex/katex that much, but I think there's no concept of "raw" rendering in katex. From what I understood, it's only made for math rendering, whereas typst can render whole documents. the "raw" mode was initially intended for rendering graphs/tables. Let me know if katex supports such things!

[cestef]

Update:

• As per @soqb suggested, math-related configuration is now grouped into the math attribute:

[markdown.math]
engine = "typst" # typst, katex, or none
svgo = true # true, false, or "path"           
css = "styles/typst-embed.css" # optional
addon = "helpers.typ" # optional

We can also simplfy specify:

[markdown]
math = "typst"

Which is equivalent to

[markdown]
math = {
	engine = "typst"
}

• A new cache configuration parameter has been added to control caching from typst or katex (and possibly more integrations in the future, thus, I put it in markdown instead of markdown.math):

[markdown]
cache = true # true, false or "path"

By default the caches are now stored in ~/.cache/zola/<CACHE_NAME>.

• CodeBlocks are now wrapped inside a CodeBlockType enum. This is due to the introduction of "renderable" code blocks:

## Wonderful test

This is markdown content.

```typ
= Hello, World!
This is a raw typst document: $y^2 = x^3 + a x + b$
```

• To allow for future additions (e.g. mathjax), a MathCompiler trait has been added.
  If raw rendering, is supported, code fence languages that it renders should be returned at raw_extensions.
  Only the specified rendering engine is now created and boxed, this means that rendering both typst and katex in the same document is not possible

• Added addons for math which allow users to add code snippets to all compiled expressions. This is useful when you need to provide helper functions, import default packages or override defaults.

PS: Thank you so much for your feedback soqdb 😄

[Keats]

I wouldn't bother with the math = "typst" form in the config

[cestef]

@soqb after benchmarking papaya and dashmap, it looks like the difference for our scale is very very small...
Benchmarks were ran on a site with around 500 elements in cache:

Hot run (read heavy):

Cold run (write heavy):

[lilymonad]

I tried this PR, and used katex, but I had to include the katex.css file myself in my base html template. We need this CSS file in order to hide the mathml output. Wasn't it the job of the css attribute ?

[cestef]

css attribute was for css injected inside the svg (typst). For KaTeX, there is nothing injected. Maybe I should update the documentation for this.