Commit 75c98d0
authored
Fix remaining AFDocs checks: HTML/MD parity, llms.txt coverage, and discoverability directives (#2929)
* llms: regenerate markdown, inject directives
Add turndown + gfm dependencies and enhance llms-html-injector to derive per-page .md files from rendered HTML (cheerio + turndown), prepend llms.txt directive to .md files, inject <link rel="alternate" type="text/markdown"> into HTML heads and an sr-only body directive, prune stale links from llms-*.txt, and generate llms-all-*.txt indexes from sitemap.xml. Also mark various DocItem UI chrome with data-markdown-ignore so HTML→MD parity ignores UI-only elements, and update static/llms.txt to reference the new complete indexes. package.json and package-lock.json updated to include the new dependencies.
* Replace turndown with node-html-markdown
Swap turndown/turndown-plugin-gfm for node-html-markdown and update llms HTML->MD pipeline. package.json/package-lock: add node-html-markdown and remove turndown deps. src/plugins/llms-html-injector: replace TurndownService with NodeHtmlMarkdown, add custom translators/options to preserve <details>/<summary> and narrow escaping for parity, change injected markdown alt link to use path-only hrefs, and add Vercel preview detection + host-rewrite logic to rewrite production URLs to the preview origin for llms*.txt and per-page .md files. src/theme/DocItem/Layout/index.jsx: change data-markdown-ignore wrapper from <span> to <div> around CopyPageButton so it is stripped correctly during MD generation.
* Prefer VERCEL_BRANCH_URL for previews
Add cheerio dependency and update preview URL handling for Vercel deployments.
resolvePreviewSiteUrl now prefers VERCEL_BRANCH_URL and falls back to VERCEL_URL, with expanded JSDoc explaining Vercel env vars and rationale. postProcessLlmsOutput logs which env var was used and rewrites build artifacts to point at the preview host so AFDocs checks (llms txt/markdown link checks) work correctly on branch/preview/development deployments.
* Regenerate MD, mark KaTeX, add Vercel header
Regenerate per-page .md files from their rendered HTML and mark KaTeX MathML spans to improve HTML/Markdown parity. Renamed and expanded the HTML->MD flow (regenerateMdFromHtml): walk build/**/index.html, convert article content to markdown, create missing .md siblings, and report regenerated/created/skipped counts. Add a regex and injection step to mark <span class="katex-mathml"> with data-markdown-ignore (idempotent) and log how many spans were marked. Improve katex handling in htmlToMarkdown by extracting the visual .katex-html text into a <code> node to avoid duplicated math content and by flattening Docusaurus chrome inside <details> so NHM output matches the rendered HTML. Tweak node-html-markdown options/translators (details/summary handling and line-start escaping) and update injector return values and logging. Also add a Vercel route header to serve llms*.txt as text/markdown with utf-8 charset.
* Route root to /llms.txt and add headers
Rewrite requests for the site root to /llms.txt in middleware because the homepage has no .md sibling (the root index.html is skipped by the llms-html-injector), ensuring the documentation index is served as text/markdown. Add a Content-Signal line to static/robots.txt to advertise search/AI allowances (search=yes, ai-input=yes, ai-train=yes). Add a Link header for "/" in vercel.json pointing to </llms.txt> as rel="service-doc" and rel="alternate"; type="text/markdown" so clients can discover the markdown service doc.
* Use internal MetaMask provider link in EVM docs
Replace external MetaMask Provider URL with the internal /metamask-connect/evm/reference/provider-api/ link across many embedded-wallets EVM React Native docs. Also include minor updates to SDK docs, plugins, example maps, static images/logos and vercel.json to keep assets and tooling in sync.
* Make /quickstart route canonical, remove splat
Register the plugin-provided /quickstart route as the single canonical page and prevent the pages plugin from auto-generating a duplicate module-less route. Exclude quickstart/index.jsx in docusaurus.config.js and remove the `${routePath}/*` splat route in the virtual-files plugin; navigation uses query strings so no catch-all child route is needed and the splat produced malformed `/quickstart/*/` sitemap entries.
* Update Algolia indexName to project name
Change the Algolia indexName in docusaurus.config.js from 'mmdocs' to 'MetaMask Documentation' to reflect the project's full name in search configuration. Verify that the corresponding Algolia index exists and that the provided appId/apiKey remain valid after this change.
* Normalize sitemap before reading URLs
Move normalizeSitemap earlier in postProcessLlmsOutput so the sitemap is normalized (dropping <loc> entries shadowed by vercel.json redirects and stripping trailing slashes from dotted-segment routes) before readSitemapUrls, pruneStaleLlmsLinks, and generateAllPagesIndex run. This ensures generated llms-all-*.txt indexes and pruning operate on the same 200-only URL set the deployed sitemap exposes and avoids pointing agents at redirecting pages. Add console logging for skipped or processed sitemap normalization.1 parent 54a22df commit 75c98d0
73 files changed
Lines changed: 1192 additions & 151 deletions
File tree
- embedded-wallets
- connect-blockchain/evm
- aleph-zero
- ancient8
- arbitrum
- astar-zkevm
- astar-zkyoto
- avalanche
- base
- bitkub
- bnb
- celo
- chiliz
- cronos
- ethereum
- fhenix
- flare
- flow
- harmony
- hedera
- kinto
- klaytn
- linea
- manta
- metis
- mint
- monad
- moonbeam
- moonriver
- morph
- neon
- nibiru
- opbnb
- optimism
- polygon
- rootstock
- saakuru
- shardeum
- skale
- soneium
- songbird
- unichain
- xdc
- zetachain
- zilliqa
- zircuit
- sdk/js
- src
- plugins
- docusaurus-plugin-virtual-files
- llms-html-injector
- theme/DocItem/Layout
- utils
- static
- img
- embedded-wallets
- authentication/facebook
- flow-diagrams
- mfa
- tutorials
- logos
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
211 | 219 | | |
212 | 220 | | |
213 | 221 | | |
| |||
655 | 663 | | |
656 | 664 | | |
657 | 665 | | |
658 | | - | |
| 666 | + | |
659 | 667 | | |
660 | 668 | | |
661 | 669 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
0 commit comments