fix(search): strip MDX syntax from search index - #3543
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c0e9ce3114
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Rsdoctor Bundle Diff AnalysisFound 5 projects in monorepo, 3 projects with changes. 📊 Quick Summary
📋 Detailed Reports (Click to expand)📁 website_htmlPath:
📦 Download Diff Report: website_html Bundle Diff 📁 website_mdPath:
📦 Download Diff Report: website_md Bundle Diff 📁 website_jsPath:
📦 Download Diff Report: website_js Bundle Diff Generated by Rsdoctor GitHub Action |
Deploying rspress-v2 with
|
| Latest commit: |
b3f86c9
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0bb019c2.rspress-v2.pages.dev |
| Branch Preview URL: | https://fix-search-index-mdx-syntax.rspress-v2.pages.dev |
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR updates search indexing to parse content as MDX, ensuring MDX-only syntax (exports, JSX attributes, expressions) is excluded while keeping user-visible text and searchable code block content.
Changes:
- Add
remark-mdxto the markdown processor so MDX nodes are parsed and can be handled correctly during indexing. - Adjust MDX node skipping/handling to keep visible JSX children while excluding MDX expressions/ESM from the index.
- Add a regression fixture + test covering MDX expressions/exports and fenced JSON code blocks.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| packages/core/src/node/route/fixtures/content-processing/with-mdx-syntax.mdx | Adds an MDX fixture containing exports, JSX, expressions, and a JSON fenced code block for regression coverage. |
| packages/core/src/node/route/extractPageData.ts | Parses MDX via remark-mdx and adds whitespace normalization after stripping MDX expressions. |
| packages/core/src/node/route/extractPageData.test.ts | Adds a regression test to ensure MDX syntax is stripped while visible content + code blocks remain searchable. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
SoonIter
changed the title
c0e9ce3 to
b3f86c9
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3f86c9594
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const createProcessor = (searchCodeBlocks: boolean) => | ||
| unified() | ||
| .use(remarkParse) | ||
| .use(remarkMdx) |
There was a problem hiding this comment.
.md pages on the Markdown parser
When a routed .md file contains raw HTML that is valid Markdown but invalid MDX/JSX—such as an HTML comment or <img> without a JSX self-closing slash—this unconditional plugin makes processor.parse(content) use MDX grammar and throw. The main compiler deliberately selects the format from the extension and enables rehypeRaw for .md in packages/core/src/node/mdx/options.ts, so these are supported pages; since createPageData awaits extraction without an error fallback, such a page can now abort a normal build. Apply remarkMdx only when processing .mdx files.
Useful? React with 👍 / 👎.
Summary
Related Issue
None.
Checklist