feat(markdown): add support for math rendering with typst and katex #2791

cestef · 2025-02-07T12:48:19Z

IMPORTANT: Please do not create a Pull Request adding a new feature without discussing it first.

The place to discuss new features is the forum: https://zola.discourse.group/
If you want to add a new feature, please open a thread there first in the feature requests section.

Sanity check:

Have you checked to ensure there aren't other open Pull Requests for the same update/change?
- Add optional math support for markdown #2708 stalled

Code changes

(Delete or ignore this section for documentation changes)

Are you doing the PR on the next branch?

If the change is a new feature or adding to/changing an existing one:

Have you created/updated the relevant documentation page(s)?

cestef · 2025-02-07T13:12:09Z

I don't really know how to get rid of these Cargo.lock updates, but anyways

cestef · 2025-02-07T13:25:17Z

This PR adds math rendering on the "server-side" with typst, while keeping the katex option open for latex.

A GenericCache struct has also been added for integrations like these to store the compilation results to save time on subsequent builds. Ideally, each integration would have its cache in the the global <Site>.caches attribute. Cache is stored as binary-encoded data at .cache/<CACHE_NAME>. Typst remote packages are also downloaded to this path.

The output SVGs generated by typst can also be minified via the svgo CLI. This is kind of a hacky implementation because it relies on the CLI but it does the job.

Newly added configuration parameters:

math = "typst"                       # Defaults to "none", one of "typst", "katex"
math_svgo = true                     # Optimize the SVGs generated by the math renderer
math_svgo_config = "svgo.config.mjs" # Optional, Path to the svgo config
math_css = "styles/typst-embed.css"  # path to the CSS file to inject inside the SVGs generated

CI is failing because [email protected] (used by typst and image) and [email protected] (used for caching keys) require rustc >= 1.81, should we upgrade rustc or downgrade these dependencies ?

The size of the binary also drastically increases because of typst's ecosystem (36.4 MiB -> 62.0 MiB)

claytonwramsey · 2025-02-11T06:06:16Z

This is great work! One possible place to improve: the current implementation outputs SVG for equations. It might be better to use MathML to make sites smaller and more accessible.

cestef · 2025-02-11T06:14:37Z

Thanks a lot :D Typst doesn't yet support HTML or MathML output (see typst/typst#5512)

The basic katex implementation however already outputs HTML I think. I don't use LaTeX that much so if anyone is up to help with this, I'd be glad.

claytonwramsey · 2025-02-11T15:53:15Z

I think it might be possible to write a backend visitor to output MathML (cf https://github.com/wcshds/typst-math-to-mathml-converter). This would be a lot easier than what the Typst team is building, since we don't have to handle everything else, but the downside is that we're then implementing a possibly buggy different spec from what Typst does.

As a stopgap for accessibility, we might also just include the math source as alt text for the generated SVGs.

cestef · 2025-02-11T16:18:22Z

This indeed sounds a bit risky, including the original source in the alt should be enough for now. I will tinker a bit with the library you linked to try out mathml rendering, I also stumbled accross this one:

https://github.com/xushengfeng/xmmath

cestef · 2025-02-18T09:46:57Z

I have tried out a bit the library you linked, here's the result I get:

I am using katex's font for the rendered mathml. There are still a lot of inconsitencies (the spaces between $a$ and $x$, the spacing feeling weird), but overall it seems like a viable solution if we were to tweak things a bit.

For reference, here's the result with SVG rendering:

soqb

this looks great! i've left a few comments but i've been toying locally for a few days and i'm very happy.

+1 for svg+mathml too. it's always what i gravitate to and it always works well. one day soon mathml will be widespread and complete enough that it will become a viable primary target, but for now, svg rendering is the only consistent option.

on that note, did you ever consider mathjax? the latex support is a little better than katex these days.

components/config/src/config/markup.rs

components/content/src/page.rs

soqb · 2025-02-14T21:14:26Z

components/markdown/src/cache.rs

+    V: Serialize + for<'de> Deserialize<'de>,
+{
+    cache_file: PathBuf,
+    cache: DashMap<K, V>,


i'm not totally sure whether it matters at the kind of scale/concurrency zola operates on, but you might find papaya a better fit for a concurrent hash map. this (biased) blogpost by the author totally convinced me.

components/markdown/src/cache.rs

components/markdown/src/markdown.rs

soqb · 2025-02-14T21:44:45Z

components/markdown/src/markdown.rs

@@ -564,7 +597,7 @@ pub fn markdown_to_html(
                        cmark::CodeBlockKind::Fenced(fence_info) => FenceSettings::new(fence_info),
                        _ => FenceSettings::new(""),
                    };
-                    let (block, begin) = match CodeBlock::new(fence, context.config, path) {
+                    let (block, begin) = match CodeBlock::new(&fence, context.config, path) {


performance wise, though it's somewhat (but not totally) moot, you might want to clone fence.language beforehand so you don't have to pay the larger cost of cloning inside CodeBlock::new.

soqb · 2025-02-14T21:47:09Z

components/markdown/src/markdown.rs

-                        accumulated_block.clear();
+                        let inner = &accumulated_block;
+                        match code_block_language.as_deref() {
+                            Some("typ") => {


is there a reason katex isn't supported here too?

I don't use latex/katex that much, but I think there's no concept of "raw" rendering in katex. From what I understood, it's only made for math rendering, whereas typst can render whole documents. the "raw" mode was initially intended for rendering graphs/tables. Let me know if katex supports such things!

components/markdown/src/markdown.rs

components/config/src/config/markup.rs

components/markdown/src/math/mod.rs

cestef · 2025-02-19T13:27:59Z

Update:

As per @soqb suggested, math-related configuration is now grouped into the math attribute:

[markdown.math]
engine = "typst" # typst, katex, or none
svgo = true # true, false, or "path"           
css = "styles/typst-embed.css" # optional
addon = "helpers.typ" # optional

We can also simplfy specify:

[markdown]
math = "typst"

Which is equivalent to

[markdown]
math = {
	engine = "typst"
}

A new cache configuration parameter has been added to control caching from typst or katex (and possibly more integrations in the future, thus, I put it in markdown instead of markdown.math):

[markdown]
cache = true # true, false or "path"

By default the caches are now stored in ~/.cache/zola/<CACHE_NAME>.

CodeBlocks are now wrapped inside a CodeBlockType enum. This is due to the introduction of "renderable" code blocks:

## Wonderful test

This is markdown content.

```typ
= Hello, World!
This is a raw typst document: $y^2 = x^3 + a x + b$
```

To allow for future additions (e.g. mathjax), a MathCompiler trait has been added.
If raw rendering, is supported, code fence languages that it renders should be returned at raw_extensions.
Only the specified rendering engine is now created and boxed, this means that rendering both typst and katex in the same document is not possible
Added addons for math which allow users to add code snippets to all compiled expressions. This is useful when you need to provide helper functions, import default packages or override defaults.

PS: Thank you so much for your feedback soqdb 😄

Keats · 2025-02-19T15:20:33Z

I wouldn't bother with the math = "typst" form in the config

cestef · 2025-02-19T18:23:32Z

@soqb after benchmarking papaya and dashmap, it looks like the difference for our scale is very very small...
Benchmarks were ran on a site with around 500 elements in cache:

Hot run (read heavy):

Cold run (write heavy):

lilymonad · 2025-03-16T13:11:21Z

I tried this PR, and used katex, but I had to include the katex.css file myself in my base html template. We need this CSS file in order to hide the mathml output. Wasn't it the job of the css attribute ?

cestef · 2025-03-16T13:25:39Z

css attribute was for css injected inside the svg (typst). For KaTeX, there is nothing injected. Maybe I should update the documentation for this.

lukasjuhrich · 2025-05-11T13:32:38Z

Hey folks, great work! I'm quite interested in this PR.
Has there been preceding feature discussion I'm unaware of, or is this intended to be a POC that remains open?

I'll gladly check out this branch and use it for now, but I was wondering if there are any plans from the maintainer's side to upstream this, and if yes what the blockers to this PR are (inb4 maintainer capacity).

Cheers!

cestef · 2025-05-11T14:40:22Z

This was intially just a PoC to fit my own needs for a blog, and I posted a PR here just in case anyone was interested. I'd gladly help you set up a clean PR to integrate it into zola!

p-r-o-g · 2025-06-23T06:56:19Z

I can try to recreate the changes without merge conflict if that would help? Is someone actively working on getting this done?

cestef · 2025-06-26T08:40:20Z

Merge conflicts were resolved

cestef · 2025-06-26T08:44:11Z

Hey @Keats! Any thoughts on this math rendering PR? Would love to get your feedback to move this forward 🙂

cappuccinocosmico · 2025-06-28T01:39:43Z

Hey, just a random new user passing through. This PR looks great! For me getting the ability to render typst would be really helpful for some of my use cases. Mainly because typst has so much more functionality for rendering complex mathematics compared to regular katex:

Most of the diagram generation seems to work within math mode, and based on my reading of the PR you should be able to make diagrams with something like this

# Markdown

Here *is* some **example** markdown code:
$#diagram(
	spacing: 8pt,
	cell-size: (8mm, 10mm),
	edge-stroke: 1pt,
	edge-corner-radius: 5pt,
	mark-scale: 70%,

	blob((0,1), [Add & Norm], tint: yellow, shape: hexagon),
	edge(),
	blob((0,2), [Multi-Head\ Attention], tint: orange),
	blob((0,4), [Input], shape: house.with(angle: 30deg),
		width: auto, tint: red),

	for x in (-.3, -.1, +.1, +.3) {
		edge((0,2.8), (x,2.8), (x,2), "-|>")
	},
	edge((0,2.8), (0,4)),

	edge((0,3), "l,uu,r", "--|>"),
	edge((0,1), (0, 0.35), "r", (1,3), "r,u", "-|>"),
	edge((1,2), "d,rr,uu,l", "--|>"),

	blob((2,0), [Softmax], tint: green),
	edge("<|-"),
	blob((2,1), [Add & Norm], tint: yellow, shape: hexagon),
	edge(),
	blob((2,2), [Feed\ Forward], tint: blue),
)$

Rendering these would require some modifications since you would have to include some packages from typst universe, but that is something I can totally volunteer to work on as soon as this gets merged.

cestef · 2025-06-28T16:35:07Z

Hey, thank you so much for taking interest in this PR ! It is already possible to include packages by default via the addon field in the config:

[markdown.math]
engine = "typst"
svgo = "svgo.config.mjs"
css = "styles/typst-embed.css"
addon = "helpers.typ"

This will include the helpers.typ file for all math blocks. There, you can import your packages, change the default layout, define helper functions, etc.

What you're trying to achieve can also be done via a typ code block, which will be rendered as "raw" typst:

```typ

= Hello, World!

This is a normal typst file 

$ a + b = 42 $
```

cestef force-pushed the feature/math-rendering branch 2 times, most recently from bad4e13 to 5d8d725 Compare February 7, 2025 13:09

cestef mentioned this pull request Feb 10, 2025

Non-md content formats #2790

Open

cestef changed the title ~~feat(markdown): add support for math rendering with typst~~ feat(markdown): add support for math rendering with typst and katex Feb 11, 2025

Keats force-pushed the next branch from e661fd8 to 8543c29 Compare February 14, 2025 21:57

cestef force-pushed the feature/math-rendering branch from 2cad380 to d276f92 Compare February 18, 2025 07:52

cestef and others added 7 commits February 18, 2025 08:54

feat(markdown): math rendering with typst

6f3df32

fix: remove useless ENABLE_GFM option

f4373cc

fix: extra closing tags being added in codeblocks

c62de4c

fix: allow inline html for summaries

47c4852

fix: formatting

e64b7b2

fix: math engine none panic

a2760b9

feat: support cache_dir to control where the cache is saved for math

6b6448b

cestef force-pushed the feature/math-rendering branch from d276f92 to 4d37093 Compare February 18, 2025 07:57

cestef and others added 9 commits February 18, 2025 11:00

docs: start writing docs for math rendering

4444203

feat(markdown): add basic tests for typst

1b595d2

feat(markdown): cache typst packages to the correct directory

0a33bda

feat: basic katex rendering

242f974

feat: complete katex rendering + a bit of refactoring

8f2bc47

fix: katex test

052b432

fix: switch to duktape js engine for katex

733ea26

chore: update Cargo.lock

a97d342

chore: update Cargo.lock

df62c23

cestef force-pushed the feature/math-rendering branch from 4d37093 to df62c23 Compare February 18, 2025 10:01

cestef marked this pull request as ready for review February 18, 2025 10:04

soqb reviewed Feb 19, 2025

View reviewed changes

cestef added 7 commits February 19, 2025 12:37

refactor: math configuration

3c84dd8

fix: don't use nightly feature flags

3cd9bd0

fix: set the correct cache depending on the engine

cf3af3a

refactor: create cache only on write

54a444e

fix: don't build codeblock for rendered codeblocks

51ba6d9

fix: deserialize HashMap instead of Option<HashMap>

72a3517

feat: support math = "engine"

fd349a4

fix: infinite recursion when deserializing config

1ec97e3

cestef and others added 5 commits February 19, 2025 16:27

refactor: remove CodeBlockType

3ed02a5

refactor: remove math = "engine" syntax support

40c79a2

docs: update math documentation

7ba158a

docs: typo

ec4bc5c

Merge branch 'next' into feature/math-rendering

ee72c04

Merge branch 'next' into feature/math-rendering

b93c762

feat(markdown): add support for math rendering with typst and katex #2791

Are you sure you want to change the base?

feat(markdown): add support for math rendering with typst and katex #2791

Uh oh!

Conversation

cestef commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code changes

Uh oh!

cestef commented Feb 7, 2025

Uh oh!

cestef commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claytonwramsey commented Feb 11, 2025

Uh oh!

cestef commented Feb 11, 2025

Uh oh!

claytonwramsey commented Feb 11, 2025

Uh oh!

cestef commented Feb 11, 2025

Uh oh!

cestef commented Feb 18, 2025

Uh oh!

soqb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

soqb Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

soqb Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

soqb Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

cestef Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cestef commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Keats commented Feb 19, 2025

Uh oh!

cestef commented Feb 19, 2025

Uh oh!

lilymonad commented Mar 16, 2025

Uh oh!

cestef commented Mar 16, 2025

Uh oh!

lukasjuhrich commented May 11, 2025

Uh oh!

cestef commented May 11, 2025

Uh oh!

p-r-o-g commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cestef commented Jun 26, 2025

Uh oh!

cestef commented Jun 26, 2025

Uh oh!

cappuccinocosmico commented Jun 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cestef commented Jun 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

cestef commented Feb 7, 2025 •

edited

Loading

cestef commented Feb 7, 2025 •

edited

Loading

soqb left a comment •

edited

Loading

cestef commented Feb 19, 2025 •

edited

Loading

p-r-o-g commented Jun 23, 2025 •

edited

Loading

cappuccinocosmico commented Jun 28, 2025 •

edited

Loading

cestef commented Jun 28, 2025 •

edited

Loading