Skip to content

feat: A native pipeline for content #16149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Princesseuh wants to merge 30 commits into
base: next
Choose a base branch
from
Open

feat: A native pipeline for content #16149

Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9b81609
feat(mdx): Rust MDX via satteri
Princesseuh May 11, 2026
bcce353
feat: make Markdown pluggable
Princesseuh May 12, 2026
f6b7872
fix: clean deps
Princesseuh May 12, 2026
4d0a4c4
fix: tests
Princesseuh May 12, 2026
5e54e36
fix: lint
Princesseuh May 12, 2026
d9a0f66
fix: tests
Princesseuh May 12, 2026
8a4a7ef
fix: don't commit notes
Princesseuh May 12, 2026
838a475
fix: test assert
Princesseuh May 12, 2026
c204fd0
Merge branch 'next' into feat/erika-rust-mdx
Princesseuh May 12, 2026
ac179d1
refactor: move things where they make sense
Princesseuh May 13, 2026
d74c82e
chore: remove unused deps
Princesseuh May 13, 2026
a03e004
feat: update Sätteri
Princesseuh May 18, 2026
857f322
fix: update deps
Princesseuh May 18, 2026
e9e0a6e
fix: don't blow up memory all over
Princesseuh May 19, 2026
d15120a
Merge branch 'next' into feat/erika-rust-mdx
Princesseuh May 19, 2026
a1e593a
chore: changeset
Princesseuh May 19, 2026
a108c43
fix: tests
Princesseuh May 19, 2026
8857e17
fix: update satteri
Princesseuh May 19, 2026
1fbfd6e
feat: deprecate other properties
Princesseuh May 19, 2026
a7fa4ad
Merge remote-tracking branch 'origin/next' into feat/erika-rust-mdx
Princesseuh May 20, 2026
d26ddd4
fix: test
Princesseuh May 20, 2026
f8e5d19
fix: just disable smartpunctuation actually
Princesseuh May 20, 2026
8ab9eac
refactor: and tests
Princesseuh May 21, 2026
74c5832
Merge branch 'next' into feat/erika-rust-mdx
Princesseuh May 21, 2026
7853a2f
chore: clean up
Princesseuh May 21, 2026
209b79b
fix: build
Princesseuh May 21, 2026
7c5521f
chore: update satteri
Princesseuh May 21, 2026
0afe01f
refactor: deduplicate code
Princesseuh May 21, 2026
6827983
fix: format
Princesseuh May 21, 2026
1f15ab6
fix: address comments
Princesseuh May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .changeset/mdx-processor-option.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
'@astrojs/mdx': major
---

`@astrojs/mdx` now picks up the processor from `markdown.processor` automatically, so a single Sätteri or unified configuration drives both `.md` and `.mdx` files. To run `.mdx` files through a different processor (or the same processor with different options) than your `.md` files, pass the `processor` option to the integration:

```js
// astro.config.mjs
import { defineConfig, satteri } from 'astro/config';
import mdx from '@astrojs/mdx';
import { unified } from '@astrojs/markdown-remark';

export default defineConfig({
markdown: { processor: satteri() },
integrations: [mdx({ processor: unified({ remarkPlugins: [/* ... */] }) })],
Copy link
Copy Markdown
Contributor

@matthewp matthewp May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing a little weird here, if you wanted to use unified for both, would that mean you'd do:

import { defineConfig, satteri } from 'astro/config';
import mdx from '@astrojs/mdx';
import { unified } from '@astrojs/markdown-remark';

export default defineConfig({
  markdown: { processor: unified({ remarkPlugins: [/* ... */] }) },
  integrations: [mdx({ processor: unified({ remarkPlugins: [/* ... */] }) })],
});

Or would you keep the processor in a variable and reference it in both places? Do they need to share state?

Copy link
Copy Markdown
Member Author

@Princesseuh Princesseuh May 22, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's the same as the current options, MDX inherits the options from markdown by default: https://docs.astro.build/en/guides/integrations-guide/mdx/#extendmarkdownconfig

It's a weird behavior of MDX in my opinion... but in line with what we currently have, at least. They don't need to share state, the things that could be shared are always module-level (e.g. Shiki)

});
```
58 changes: 58 additions & 0 deletions .changeset/seven-emus-knock.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
'astro': major
'@astrojs/markdown-remark': major
'@astrojs/markdown-satteri': major
---

Adds a `markdown.processor` config option to configure the Markdown processor to use. By default, [Sätteri](https://github.com/bruits/satteri) a fast Rust-based Markdown/MDX compiler is used. Both `.md` and `.mdx` files now render through Sätteri.

To pass Sätteri plugins or enable additional parser features, use the `satteri()` explicitely:

```js
// astro.config.mjs
import { defineConfig, satteri } from 'astro/config';

export default defineConfig({
markdown: {
processor: satteri({
hastPlugins: [myPlugin],
features: { directive: true, smartPunctuation: false },
}),
},
});
```

To keep using your existing remark/rehype plugins, install `@astrojs/markdown-remark` into your project and use the `unified` export:

```sh
pnpm add @astrojs/markdown-remark
```

```js
// astro.config.mjs
import { defineConfig } from 'astro/config';
import { unified } from '@astrojs/markdown-remark';
import remarkToc from 'remark-toc';

export default defineConfig({
markdown: {
processor: unified({ remarkPlugins: [remarkToc] }),
},
});
```

The top-level `markdown.remarkPlugins`, `markdown.rehypePlugins`, and `markdown.remarkRehype` options are deprecated. They'll continue to work when `@astrojs/markdown-remark` is installed for now, but this will be removed in the next major.

The top-level `markdown.gfm` and `markdown.smartypants` options are also deprecated. Move them onto your processor instead — `satteri({ features: { gfm: false, smartPunctuation: false } })`, or `unified({ gfm: false, smartypants: false })`:
Copy link
Copy Markdown
Member Author

@Princesseuh Princesseuh May 21, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, those two options could be fully removed and we could ask users to manually install the Remark plugins, but I figured for a better migration story right now we can just support them in the processor itself.

It's not relevant for Sätteri because it's supported natively there.


```js
// astro.config.mjs
import { defineConfig } from 'astro/config';
import { unified } from '@astrojs/markdown-remark';

export default defineConfig({
markdown: {
processor: unified({ gfm: false, smartypants: false }),
},
});
```
8 changes: 8 additions & 0 deletions knip.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,11 @@ export default {
// False positive because of cloudflare:workers
ignoreDependencies: ['cloudflare'],
},
'packages/integrations/mdx': {
entry: [srcEntry, dtsEntry, testEntry],
// Optional peer dep, only loaded when the user picks the unified processor
ignoreDependencies: ['@astrojs/markdown-remark'],
},
'packages/integrations/netlify': {
entry: [srcEntry, dtsEntry, testEntry],
},
Expand All @@ -85,6 +90,9 @@ export default {
'packages/markdown/remark': {
entry: [srcEntry, dtsEntry, testEntry],
},
'packages/markdown/satteri': {
entry: [srcEntry, dtsEntry, testEntry],
},
'packages/upgrade': {
entry: ['src/index.ts', testEntry],
},
Expand Down
2 changes: 1 addition & 1 deletion packages/astro/components/Code.astro
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
import { createShikiHighlighter, type ThemePresets } from '@astrojs/markdown-remark/shiki';
import { createShikiHighlighter, type ThemePresets } from '@astrojs/internal-helpers/shiki';
import type { ShikiTransformer, ThemeRegistration, ThemeRegistrationRaw } from 'shiki';
import { bundledLanguages } from 'shiki/langs';
import type { CodeLanguage } from '../dist/types/public/common.js';
Expand Down
110 changes: 55 additions & 55 deletions packages/astro/e2e/nested-in-svelte.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,88 +14,88 @@ test.afterAll(async () => {
});

test.describe('Nested Frameworks in Svelte', () => {
test('React counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));
test('React counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));

const counter = page.locator('#react-counter');
await expect(counter, 'component is visible').toBeVisible();
const counter = page.locator('#react-counter');
await expect(counter, 'component is visible').toBeVisible();

const count = counter.locator('#react-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');
const count = counter.locator('#react-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');

await waitForHydrate(page, counter);
await waitForHydrate(page, counter);

const increment = counter.locator('#react-counter-increment');
await increment.click();
const increment = counter.locator('#react-counter-increment');
await increment.click();

await expect(count, 'count incremented by 1').toHaveText('1');
});
await expect(count, 'count incremented by 1').toHaveText('1');
});

test('Preact counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));
test('Preact counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));

const counter = page.locator('#preact-counter');
await expect(counter, 'component is visible').toBeVisible();
const counter = page.locator('#preact-counter');
await expect(counter, 'component is visible').toBeVisible();

const count = counter.locator('#preact-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');
const count = counter.locator('#preact-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');

await waitForHydrate(page, counter);
await waitForHydrate(page, counter);

const increment = counter.locator('#preact-counter-increment');
await increment.click();
const increment = counter.locator('#preact-counter-increment');
await increment.click();

await expect(count, 'count incremented by 1').toHaveText('1');
});
await expect(count, 'count incremented by 1').toHaveText('1');
});

test('Solid counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));
test('Solid counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));

const counter = page.locator('#solid-counter');
await expect(counter, 'component is visible').toBeVisible();
const counter = page.locator('#solid-counter');
await expect(counter, 'component is visible').toBeVisible();

const count = counter.locator('#solid-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');
const count = counter.locator('#solid-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');

await waitForHydrate(page, counter);
await waitForHydrate(page, counter);

const increment = counter.locator('#solid-counter-increment');
await increment.click();
const increment = counter.locator('#solid-counter-increment');
await increment.click();

await expect(count, 'count incremented by 1').toHaveText('1');
});
await expect(count, 'count incremented by 1').toHaveText('1');
});

test('Vue counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));
test('Vue counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));

const counter = page.locator('#vue-counter');
await expect(counter, 'component is visible').toBeVisible();
const counter = page.locator('#vue-counter');
await expect(counter, 'component is visible').toBeVisible();

const count = counter.locator('#vue-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');
const count = counter.locator('#vue-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');

await waitForHydrate(page, counter);
await waitForHydrate(page, counter);

const increment = counter.locator('#vue-counter-increment');
await increment.click();
const increment = counter.locator('#vue-counter-increment');
await increment.click();

await expect(count, 'count incremented by 1').toHaveText('1');
});
await expect(count, 'count incremented by 1').toHaveText('1');
});

test('Svelte counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));
test('Svelte counter', async ({ astro, page }) => {
await page.goto(astro.resolveUrl('/'));

const counter = page.locator('#svelte-counter');
await expect(counter, 'component is visible').toBeVisible();
const counter = page.locator('#svelte-counter');
await expect(counter, 'component is visible').toBeVisible();

const count = counter.locator('#svelte-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');
const count = counter.locator('#svelte-counter-count');
await expect(count, 'initial count is 0').toHaveText('0');

await waitForHydrate(page, counter);
await waitForHydrate(page, counter);

const increment = counter.locator('#svelte-counter-increment');
await increment.click();
const increment = counter.locator('#svelte-counter-increment');
await increment.click();

await expect(count, 'count incremented by 1').toHaveText('1');
});
await expect(count, 'count incremented by 1').toHaveText('1');
});
});
9 changes: 4 additions & 5 deletions packages/astro/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
},
"exports": {
".": "./dist/index.js",
"./markdown": "./dist/markdown/index.js",
"./env": "./env.d.ts",
"./env/runtime": "./dist/env/runtime.js",
"./env/setup": "./dist/env/setup.js",
Expand All @@ -39,7 +40,6 @@
"./astro-jsx": "./astro-jsx.d.ts",
"./tsconfigs/*.json": "./tsconfigs/*",
"./tsconfigs/*": "./tsconfigs/*.json",
"./jsx/rehype.js": "./dist/jsx/rehype.js",
"./jsx-runtime": {
"types": "./jsx-runtime.d.ts",
"default": "./dist/jsx-runtime/index.js"
Expand Down Expand Up @@ -135,7 +135,7 @@
"dependencies": {
"@astrojs/compiler-rs": "^0.1.10",
"@astrojs/internal-helpers": "workspace:*",
"@astrojs/markdown-remark": "workspace:*",
"@astrojs/markdown-satteri": "workspace:*",
"@astrojs/telemetry": "workspace:*",
"@capsizecss/unpack": "^4.0.0",
"@clack/prompts": "^1.1.0",
Expand Down Expand Up @@ -194,6 +194,7 @@
},
"devDependencies": {
"@astrojs/check": "workspace:*",
"@astrojs/markdown-remark": "workspace:*",
"@playwright/test": "1.58.2",
"@types/aria-query": "^5.0.4",
"@types/hast": "^3.0.4",
Expand All @@ -210,8 +211,6 @@
"expect-type": "^1.3.0",
"fs-fixture": "^2.13.0",
"hono": "^4.12.14",
"mdast-util-mdx": "^3.0.0",
"mdast-util-mdx-jsx": "^3.2.0",
"node-mocks-http": "^1.17.2",
"parse-srcset": "^1.0.2",
"rehype-autolink-headings": "^7.1.0",
Expand All @@ -232,6 +231,7 @@
"publishConfig": {
"exports": {
".": "./dist/index.js",
"./markdown": "./dist/markdown/index.js",
"./env": "./env.d.ts",
"./env/runtime": "./dist/env/runtime.js",
"./env/setup": "./dist/env/setup.js",
Expand All @@ -240,7 +240,6 @@
"./astro-jsx": "./astro-jsx.d.ts",
"./tsconfigs/*.json": "./tsconfigs/*",
"./tsconfigs/*": "./tsconfigs/*.json",
"./jsx/rehype.js": "./dist/jsx/rehype.js",
"./jsx-runtime": {
"types": "./jsx-runtime.d.ts",
"default": "./dist/jsx-runtime/index.js"
Expand Down
1 change: 1 addition & 0 deletions packages/astro/src/config/entrypoint.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export { defineConfig, getViteConfig } from './index.js';
export { sessionDrivers } from '../core/session/drivers.js';
export { svgoOptimizer } from '../assets/svg/svgo.js';
export { logHandlers } from '../core/logger/handlers.js';
export { satteri } from '@astrojs/markdown-satteri';

/**
* Return the configuration needed to use the Sharp-based image service
Expand Down
18 changes: 12 additions & 6 deletions packages/astro/src/content/content-layer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
import { existsSync, promises as fs } from 'node:fs';
import {
createMarkdownProcessor,
parseFrontmatter,
type MarkdownProcessor,
} from '@astrojs/markdown-remark';
import { parseFrontmatter } from '@astrojs/internal-helpers/frontmatter';
import type { MarkdownProcessor } from '@astrojs/internal-helpers/markdown';
import PQueue from 'p-queue';
import type { FSWatcher } from 'vite';
import xxhash from 'xxhash-wasm';
Expand Down Expand Up @@ -154,7 +151,16 @@ export class ContentLayer {
content: string,
options?: RenderMarkdownOptions,
): Promise<RenderedContent> {
this.#markdownProcessor ??= await createMarkdownProcessor(this.#settings.config.markdown);
if (!this.#markdownProcessor) {
const { markdown, image } = this.#settings.config;
this.#markdownProcessor = await markdown.processor.createRenderer({
image,
syntaxHighlight: markdown.syntaxHighlight,
shikiConfig: markdown.shikiConfig,
gfm: markdown.gfm,
smartypants: markdown.smartypants,
});
}
const { frontmatter, content: body } = parseFrontmatter(content);
const { code, metadata } = await this.#markdownProcessor.render(body, {
frontmatter,
Expand Down
2 changes: 1 addition & 1 deletion packages/astro/src/content/data-store.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { MarkdownHeading } from '@astrojs/markdown-remark';
import type { MarkdownHeading } from '@astrojs/internal-helpers/markdown';
import * as devalue from 'devalue';

export interface RenderedContent {
Expand Down
6 changes: 4 additions & 2 deletions packages/astro/src/content/runtime.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { MarkdownHeading } from '@astrojs/markdown-remark';
import type { MarkdownHeading } from '@astrojs/internal-helpers/markdown';
import { escape } from 'html-escaper';
import { Traverse } from 'neotraverse/modern';
import * as z from 'zod/v4';
Expand Down Expand Up @@ -461,7 +461,9 @@ async function updateImageReferencesInBody(html: string, fileName: string) {
// function because getImage is async.
for (const [_full, imagePath] of html.matchAll(CONTENT_LAYER_IMAGE_REGEX)) {
try {
const decodedImagePath = JSON.parse(imagePath.replaceAll('&#x22;', '"'));
const decodedImagePath = JSON.parse(
imagePath.replace(/&(?:#x22|quot);/g, '"').replace(/&(?:#x27|apos);/g, "'"),
Copy link
Copy Markdown
Member Author

@Princesseuh Princesseuh May 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a remark quirk, it uses numeric character references only, but a lot of Markdown processors (and Sätteri) I've noticed tend to use named references instead. Either way, both are allowed in the Markdown spec.

Copy link
Copy Markdown
Contributor

@matthewp matthewp May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment on what this is doing?

);

let image: GetImageResult;
if (URL.canParse(decodedImagePath.src)) {
Expand Down
2 changes: 1 addition & 1 deletion packages/astro/src/content/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import fsMod from 'node:fs';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { parseFrontmatter } from '@astrojs/markdown-remark';
import { parseFrontmatter } from '@astrojs/internal-helpers/frontmatter';
import { slug as githubSlug } from 'github-slugger';
import colors from 'piccolore';
import type { RunnableDevEnvironment, Rolldown } from 'vite';
Expand Down
13 changes: 13 additions & 0 deletions packages/astro/src/core/config/merge.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,19 @@ function mergeConfigRecursively(
merged[key] = value;
continue;
}
// Markdown processor descriptors (satteri(), unified(), third-party) are opaque
// objects with a `createRenderer` function — treat them as atomic so swapping
// processors via updateConfig replaces cleanly instead of merging fields from
// the previous descriptor into the new one.
if (
key === 'processor' &&
rootPath === 'markdown' &&
isObject(value) &&
typeof value.createRenderer === 'function'
) {
merged[key] = value;
continue;
}
if (isObject(existing) && isObject(value)) {
merged[key] = mergeConfigRecursively(existing, value, rootPath ? `${rootPath}.${key}` : key);
continue;
Expand Down
Loading
Loading