Improve docs Agent Score: LLM outputs, orphan-route cleanup, and URL formatting (#2926)

* Generate LLMS outputs and add HTML injector

Enable generation of site-wide llms.txt/llms-full.txt and per-page markdown (generateMarkdownFiles) so LLM tooling can index docs. Add a new llms-html-injector plugin to normalize generated .md paths, rewrite llms* URLs, and inject <link rel="alternate" type="text/markdown"> tags. Add a verification script (scripts/verify-llms-output.js), ignore its output in .gitignore (/.llms-verify), and remove the hand-maintained static/llms.txt in favor of generated rootContent/fullRootContent. Also update config include/exclude patterns and add related docs/service index updates; package dependencies were bumped (Docusaurus packages) and a new plugin implementation was added at src/plugins/llms-html-injector/index.js.

* Ignore .llms-verify and fix LLMS file read

Add .llms-verify/** to docusaurus ignore patterns and update scripts/verify-llms-output.js to read the LLMS output file once as a buffer (avoiding a TOCTOU race). Derive byte length from the buffer, convert to UTF-8 text, and use that text for all regex counts instead of calling fs.stat and multiple reads.

* Add /.llms-inspect to .gitignore

Ignore the .llms-inspect file by adding it to .gitignore. This prevents inspection output from LLMS tooling (similar to /.llms-verify) from being accidentally committed.

* Refactor LLMs generation and add static root index

Switch from auto-generating a monolithic llms.txt/llms-full.txt to a hand-curated static/llms.txt and per-section indexes. Wrap docusaurus-plugin-llms with an llms-html-injector wrapper so generation runs before post-processing (normalizing .md layout, rewriting llms*.txt URLs, and injecting <link rel="alternate">). Update docusaurus.config.js to disable the classic docs slot, disable root/full LLM generators, and adjust path transformations. Update scripts/verify-llms-output.js to mirror the new behavior (customLLMFiles list, disabled root generators, post-process step, and copy static/llms.txt into the verify outDir). Add static/llms.txt, add vercel redirects for the removed whats-new page, remove docs/whats-new.md, and tweak a navbar CSS comment. Export postProcessLlmsOutput for reuse by the verify script.

* Probe target existence before rename

Use fs.access as an explicit existence probe and distinguish an absent target from real I/O failures. The change introduces a targetExists flag and a try/catch around fs.access so that expected "not found" control flow proceeds to fs.rename, while genuine errors from fs.unlink (e.g. permission issues or locks) will propagate instead of being silently swallowed. A clarifying comment was also added to explain the reasoning.

* Docs: normalize URLs & disable orphan pages

Disable the default blog and the standalone Algolia search page and exclude quickstart builder fragments from the pages plugin to prevent orphan routes (Ahrefs orphan report). Replace CopyableNoFollow usages with inline/backticked RPC and explorer URLs across many MDX files and normalize links (e.g. web3.js → official docs, Nethereum guide URL). Update Ethereum Sepolia RPC to publicnode, remove src/pages/developer-tools/index.tsx, and apply minor layout/vercel config tweaks.

* Wrap RPC and explorer URLs in backticks

Update formatting in numerous embedded-wallets/connect-blockchain MDX files to wrap Public RPC URL and Block Explorer Link values in inline code (backticks) for consistent rendering across docs. Purely documentation/formatting changes with no functional code modifications.

* Wrap llms plugin to forward hooks and postBuild

Spread the upstream plugin object into the wrapper so any existing or future lifecycle hooks (e.g. contentLoaded, loadContent, configureWebpack, etc.) are forwarded unchanged. Override name to 'llms-html-injector' and override postBuild to first invoke the upstream postBuild using .call(inner, props) to preserve its context, then run postProcessLlmsOutput(props.outDir). Adds a clarifying comment explaining the forwarding rationale.

* Support non-root baseUrl when rewriting LLMS URLs

Compute and propagate the resolved site URL (url + baseUrl) so LLMS index rewrites and injected <link rel="alternate"> hrefs match deployments under a non-root base path. Added resolveSiteUrl(siteConfig) and require a siteUrl in postProcessLlmsOutput, then threaded siteUrl through rewriteLlmsIndexes and injectAlternateLinks. Updated the verify script to mirror docusaurus baseUrl resolution (honoring DEST env), import resolveSiteUrl, and pass the computed siteUrl to the post-processing step; also improved logging to show the siteUrl used. Includes a runtime check to fail if postProcessLlmsOutput is called without a siteUrl.

* Prefer MDX named exports for media metadata

Replace YAML front matter with JS named exports (contentType, mediaContent) in two quickstart MDX files and update toStep to prefer those named exports with frontMatter as a fallback. Also add typings and explanatory comments to Module so partial MDX imports (which can't carry front matter in Docusaurus 3.x) can provide contentType/mediaContent via exports.

* Remove Wallet LLMS entries from config and script

Delete the llms-wallet.txt and llms-wallet-full.txt entries from docusaurus.config.js and scripts/verify-llms-output.js. These entries previously configured generation of Wallet API LLMS outputs (short and full); removing them stops creation of those files and simplifies the LLMS configuration.

* Split Embedded Wallets LLMS and add middleware

Split the large llms-embedded-wallets full file into multiple smaller, sub-domain LLMS files (SDK, EVM, non-EVM, platform) and update docusaurus.config.js, the LLMS verification script, and static/llms.txt links to reflect the split to keep each file within LLM context limits. Add a Vercel Edge middleware (middleware.ts) to perform Accept: text/markdown content negotiation and rewrite requests to the corresponding .md siblings (replacing the previous vercel.json rewrite rule). Update vercel.json to remove the rewrites and add redirects for legacy llms wallet paths. Add @vercel/edge to package.json. Also fix minor docs formatting (remove trailing slashes from Monad block explorer links).

* Extract LLMS plugin options to module

Move llms plugin configuration into a single CommonJS module at src/plugins/llms-html-injector/options.js and make it the source of truth for both the production build and the verify script. docusaurus.config.js now requires and passes the shared options to the llms-html-injector plugin, and scripts/verify-llms-output.js imports the same options instead of duplicating arrays/objects. This eliminates silent-drift between the build and sanity-check logic and centralizes ignoreFiles, customLLMFiles, pathTransformation, and related flags (generateLLMsTxt, generateLLMsFullTxt, generateMarkdownFiles, etc.). The new module exports the full options object plus helper exports for customLLMFiles and ignoreFiles.

* Export plugin options as default object

Export the llmsPluginOptions object directly (module.exports = llmsPluginOptions) and remove the individual property exports. This avoids adding a self-referential property that would create a circular structure and crash Docusaurus' genSiteConfig during JSON.stringify; customLLMFiles and ignoreFiles remain accessible as properties on the exported object.

* docs: minor formatting and copy fixes

Apply small documentation improvements across embedded-wallets and tutorials: wrap tickers and scopes in code formatting, standardize MetaMask and opBNB casing, fix punctuation and spacing (including duplicate 'by'), consolidate an img tag into one line, adjust phrasing ('for free' -> 'at no cost'), and update Wagmi TabItem value to 'wagmi-tab' for clarity. These changes improve consistency and readability of the docs.

* Support .mdx and improve llms injector logging

Include .mdx in services LLMS globs so reference pages are not excluded; update llms-html-injector to warn when two source markdown files normalize to the same target, return/count a 'root' sentinel for intentionally skipped site-root index.html pages and surface that in the summary log. Add explanatory comments in docusaurus.config about the unused docs slot and Vercel cleanUrls behavior. Add Vercel redirects for /blog and /search to prevent orphaned routes.

Author: shahbaz17

.gitignore

@@ -4,6 +4,10 @@
 # Production
 /build
 
+# Dev sanity-check output from scripts/verify-llms-output.js
+/.llms-verify
+/.llms-inspect
+
 # Generated files
 .docusaurus
 .cache-loader

docs/whats-new.md

@@ -10,6 +10,14 @@ const productsDropdown = fs.readFileSync('./src/components/NavDropdown/Products.
 const baseUrl = process.env.DEST || '/'
 const siteUrl = 'https://docs.metamask.io'
 
+// Options for the `llms-html-injector` plugin (which wraps
+// `docusaurus-plugin-llms`). Centralized in a standalone CommonJS module so
+// the same configuration drives both this production build and the
+// `scripts/verify-llms-output.js` sanity check, eliminating the silent-drift
+// risk that existed when both files maintained their own copy of the
+// `customLLMFiles` array and `ignoreFiles` list.
+const llmsPluginOptions = require('./src/plugins/llms-html-injector/options')
+
 const remarkPlugins = [
   require('remark-math'),
   [require('@docusaurus/remark-plugin-npm2yarn'), { sync: true }],
@@ -162,16 +170,23 @@ const config = {
     [
       'classic',
       {
-        docs: {
-          id: 'docs',
-          path: 'docs',
-          routeBasePath: '/',
-          editUrl: 'https://github.com/MetaMask/metamask-docs/edit/main/',
-          sidebarPath: false,
-          breadcrumbs: false,
-          remarkPlugins,
-          rehypePlugins,
-        },
+        // The classic preset's `docs` plugin is disabled. It previously
+        // rendered a single orphan page (`docs/whats-new.md`) at `/whats-new/`
+        // that pointed offsite for release notes. The page had no incoming
+        // internal links and is redirected to `/` in `vercel.json`. Per-product
+        // documentation is served by dedicated `plugin-content-docs` instances
+        // configured below in `plugins`, not by this preset slot.
+        //
+        // Theme-safety note: theme overrides under `src/theme/DocItem/**`
+        // consume `useDoc` from `@docusaurus/plugin-content-docs/client`,
+        // which binds to whichever named content-docs instance owns the
+        // current route. The classic preset's docs slot is unused at runtime,
+        // so disabling it has no impact on theme rendering.
+        docs: false,
+        // The blog plugin is disabled too: this site has no blog content and
+        // the default-enabled `/blog/` route was being indexed as an orphan
+        // (Ahrefs orphan report, 2026-05-25).
+        blog: false,
         pages: {
           path: 'src/pages',
           routeBasePath: '/',
@@ -181,6 +196,18 @@ const config = {
             '**/_*/**',
             '**/*.test.{js,jsx,ts,tsx}',
             '**/__tests__/**',
+            // The quickstart "builder" tree is a library of React/MDX content
+            // fragments imported by `src/pages/quickstart/index.jsx`. The default
+            // `**/**.{js,jsx,ts,tsx,md,mdx}` include also turns each fragment
+            // into a standalone, content-less route (Ahrefs orphan report,
+            // 2026-05-25). Excluding the trees here removes the routes without
+            // breaking the imports.
+            'quickstart/builder/**',
+            'quickstart/commonSteps/**',
+            'quickstart/NavigationOverlay/**',
+            'quickstart/MediaStep/**',
+            'quickstart/interfaces.ts',
+            'quickstart/utils.tsx',
           ],
           mdxPageComponent: '@theme/MDXPage',
           remarkPlugins,
@@ -323,164 +350,28 @@ const config = {
         containerId: 'GTM-5FGPLC2Q',
       },
     ],
-    [
-      'docusaurus-plugin-llms',
-      {
-        // Set docsDir to site root to collect from all directories
-        docsDir: '.',
-        // Disable default files since we're generating section-specific files
-        generateLLMsTxt: false,
-        generateLLMsFullTxt: false,
-        // Ignore common non-doc directories
-        // Note: src/pages/** is not ignored so tutorials can be collected
-        // Each customLLMFiles entry filters by includePatterns, so only matching files are included
-        ignoreFiles: [
-          'node_modules/**',
-          'build/**',
-          '.docusaurus/**',
-          'static/**',
-          'src/components/**',
-          'src/theme/**',
-          'src/lib/**',
-          'src/config/**',
-          'src/hooks/**',
-          'src/utils/**',
-          'src/plugins/**',
-          'src/specs/**',
-          'src/client/**',
-          'src/scss/**',
-          'i18n/**',
-          '*.config.js',
-          '*.json',
-          '*.lock',
-          'README.md',
-          'CONTRIBUTING.md',
-          'gator_versioned_docs/**',
-        ],
-        excludeImports: true,
-        removeDuplicateHeadings: true,
-        // Path transformation to fix URL construction
-        // Since docsDir is '.', we need to remove 'docs/' prefix and handle src/pages paths
-        pathTransformation: {
-          ignorePaths: ['docs', 'src/pages'],
-        },
-        // Generate separate files for each section
-        customLLMFiles: [
-          {
-            filename: 'llms-embedded-wallets.txt',
-            includePatterns: ['embedded-wallets/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'MetaMask Embedded Wallets documentation',
-            description: 'Documentation links for MetaMask Embedded Wallets',
-          },
-          {
-            filename: 'llms-embedded-wallets-full.txt',
-            includePatterns: ['embedded-wallets/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'MetaMask Embedded Wallets documentation',
-            description: 'Complete documentation for MetaMask Embedded Wallets',
-          },
-          {
-            filename: 'llms-metamask-connect.txt',
-            includePatterns: ['metamask-connect/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'MetaMask Connect documentation',
-            description: 'Documentation links for MetaMask Connect',
-          },
-          {
-            filename: 'llms-metamask-connect-full.txt',
-            includePatterns: ['metamask-connect/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'MetaMask Connect documentation',
-            description: 'Complete documentation for MetaMask Connect',
-          },
-          {
-            filename: 'llms-smart-accounts-kit.txt',
-            includePatterns: ['smart-accounts-kit/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'MetaMask Smart Accounts Kit documentation',
-            description: 'Documentation links for MetaMask Smart Accounts Kit',
-          },
-          {
-            filename: 'llms-smart-accounts-kit-full.txt',
-            includePatterns: ['smart-accounts-kit/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'MetaMask Smart Accounts Kit documentation',
-            description: 'Complete documentation for MetaMask Smart Accounts Kit',
-          },
-          {
-            filename: 'llms-snaps.txt',
-            includePatterns: ['snaps/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'Snaps documentation',
-            description: 'Documentation links for Snaps',
-          },
-          {
-            filename: 'llms-snaps-full.txt',
-            includePatterns: ['snaps/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'Snaps documentation',
-            description: 'Complete documentation for Snaps',
-          },
-          {
-            filename: 'llms-wallet.txt',
-            includePatterns: ['wallet/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'Wallet API documentation',
-            description: 'Documentation links for Wallet API',
-          },
-          {
-            filename: 'llms-wallet-full.txt',
-            includePatterns: ['wallet/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'Wallet API documentation',
-            description: 'Complete documentation for Wallet API',
-          },
-          {
-            filename: 'llms-tutorials.txt',
-            includePatterns: ['src/pages/tutorials/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'Tutorials',
-            description: 'Documentation links for MetaMask tutorials',
-          },
-          {
-            filename: 'llms-tutorials-full.txt',
-            includePatterns: ['src/pages/tutorials/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'Tutorials',
-            description: 'Complete documentation for MetaMask tutorials',
-          },
-          {
-            filename: 'llms-dashboard.txt',
-            includePatterns: ['developer-tools/dashboard/**/*.{md,mdx}'],
-            fullContent: false,
-            title: 'Developer dashboard documentation',
-            description: 'Documentation links for MetaMask Developer dashboard',
-          },
-          {
-            filename: 'llms-dashboard-full.txt',
-            includePatterns: ['developer-tools/dashboard/**/*.{md,mdx}'],
-            fullContent: true,
-            title: 'Developer dashboard documentation',
-            description: 'Complete documentation for MetaMask Developer dashboard',
-          },
-          {
-            filename: 'llms-services.txt',
-            includePatterns: ['services/**/*.md'],
-            fullContent: false,
-            title: 'Services documentation',
-            description: 'Documentation links for MetaMask services',
-          },
-          {
-            filename: 'llms-services-full.txt',
-            includePatterns: ['services/**/*.md'],
-            fullContent: true,
-            title: 'Services documentation',
-            description: 'Complete documentation for MetaMask services',
-          },
-        ],
-      },
-    ],
+    // The llms-html-injector plugin wraps docusaurus-plugin-llms so that
+    // generation (per-section `llms-<product>.txt` files and per-page `.md`
+    // files) runs sequentially before our post-processing (path normalization,
+    // index URL rewrites, and `<link rel="alternate" type="text/markdown">`
+    // injection). Wrapping is required because Docusaurus 3.x executes
+    // `postBuild` hooks concurrently via `Promise.all`, so registering both
+    // plugins separately would let the injector race against the generator.
+    //
+    // Options (ignoreFiles, customLLMFiles, pathTransformation, etc.) live in
+    // `./src/plugins/llms-html-injector/options.js` so the local sanity-check
+    // script can consume the exact same configuration.
+    //
+    // Related Vercel-level note (kept here because `vercel.json` is pure JSON
+    // and can't carry inline comments): `vercel.json` sets `cleanUrls: false`.
+    // With `trailingSlash: true` Docusaurus emits directory-style
+    // `/foo/index.html` URLs and never `*.html` leaves, so Vercel's `.html`
+    // stripping is a no-op for our content and adds an unnecessary redirect
+    // hop for any static `*.html` asset. Disabling it also stops Vercel from
+    // doing its own URL normalization underneath the Edge middleware in
+    // `middleware.ts`, which performs the `Accept: text/markdown`
+    // content-negotiation rewrite to the `.md` siblings emitted here.
+    ['./src/plugins/llms-html-injector', llmsPluginOptions],
   ],
   clientModules: [require.resolve('./src/client/scroll-fix.js')],
   themeConfig:
@@ -775,6 +666,10 @@ const config = {
         askAi: {
           assistantId: 'REak1eiP5wfp',
         },
+        // Disable the standalone `/search/` results page. The Algolia DocSearch
+        // modal still works; the dedicated page was being indexed as an orphan
+        // (Ahrefs orphan report, 2026-05-25).
+        searchPagePath: false,
         // Optional: see doc section below
         // Optional: Specify domains where the navigation should occur through window.location instead on history.push. Useful when our Algolia config crawls multiple documentation sites and we want to navigate with window.location.href to them.
         // externalUrlRegex: "external\\.com|domain\\.com",
@@ -785,8 +680,6 @@ const config = {
         // },
         // Optional: Algolia search parameters
         // searchParameters: {},
-        // Optional: path for search page that enabled by default (`false` to disable it)
-        // searchPagePath: 'search',
         //... other Algolia params
       },
       mermaid: {

docusaurus.config.js

@@ -11,9 +11,9 @@ import TabItem from '@theme/TabItem'
 <TabItem value="mainnet">
 
 - **Chain ID:** 0x1b5
-- **Public RPC URL:** https://rpc.azero.dev
+- **Public RPC URL:** `https://rpc.azero.dev`
 - **Display Name:** Aleph Zero Mainnet
-- **Block Explorer Link:** https://alephzero.subscan.io
+- **Block Explorer Link:** `https://alephzero.subscan.io`
 - **Ticker:** AZERO
 - **Ticker Name:** Aleph Zero
 
@@ -22,9 +22,9 @@ import TabItem from '@theme/TabItem'
 <TabItem value="testnet">
 
 - **Chain ID:** 0x1b6
-- **Public RPC URL:** https://rpc.test.azero.dev
+- **Public RPC URL:** `https://rpc.test.azero.dev`
 - **Display Name:** Aleph Zero Testnet
-- **Block Explorer Link:** https://test.azero.subscan.io
+- **Block Explorer Link:** `https://test.azero.subscan.io`
 - **Ticker:** AZERO
 - **Ticker Name:** Aleph Zero
 

embedded-wallets/connect-blockchain/_general-connect-blockchain/_aleph-zero.mdx

@@ -1,6 +1,5 @@
 import Tabs from '@theme/Tabs'
 import TabItem from '@theme/TabItem'
-import CopyableNoFollow from '@site/src/components/CopyableNoFollow'
 
 <Tabs
  defaultValue="mainnet"
@@ -12,9 +11,9 @@ import CopyableNoFollow from '@site/src/components/CopyableNoFollow'
 <TabItem value="mainnet">
 
 - **Chain ID:** 0x12c
-- **Public RPC URL:** <CopyableNoFollow url="https://rpc.ancient8.gg" />
+- **Public RPC URL:** `https://rpc.ancient8.gg`
 - **Display Name:** Ancient8 Mainnet
-- **Block Explorer Link:** [https://scan.ancient8.gg](https://scan.ancient8.gg)
+- **Block Explorer Link:** `https://scan.ancient8.gg`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum
 
@@ -23,9 +22,9 @@ import CopyableNoFollow from '@site/src/components/CopyableNoFollow'
 <TabItem value="testnet">
 
 - **Chain ID:** 0x2af3
-- **Public RPC URL:** <CopyableNoFollow url="https://rpc-testnet.ancient8.gg" />
+- **Public RPC URL:** `https://rpc-testnet.ancient8.gg`
 - **Display Name:** Ancient8 Testnet
-- **Block Explorer Link:** [https://scan-testnet.ancient8.gg](https://scan-testnet.ancient8.gg)
+- **Block Explorer Link:** `https://scan-testnet.ancient8.gg`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum
 

embedded-wallets/connect-blockchain/_general-connect-blockchain/_ancient8.mdx

@@ -13,7 +13,7 @@ import TabItem from '@theme/TabItem'
 - **Chain ID:** 0xa4b1
 - **RPC URL:** You can use our bundled RPC service from Infura, or your own choice of RPC service for production.
 - **Display Name:** Arbitrum One
-- **Block Explorer Link:** https://arbiscan.io
+- **Block Explorer Link:** `https://arbiscan.io`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum
 
@@ -22,9 +22,9 @@ import TabItem from '@theme/TabItem'
 <TabItem value="sepolia">
 
 - **Chain ID:** 0x66eee
-- **Public RPC URL:** https://sepolia-rollup.arbitrum.io/rpc
+- **Public RPC URL:** `https://sepolia-rollup.arbitrum.io/rpc`
 - **Display Name:** Arbitrum Sepolia
-- **Block Explorer Link:** https://sepolia.arbiscan.io
+- **Block Explorer Link:** `https://sepolia.arbiscan.io`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum
 

embedded-wallets/connect-blockchain/_general-connect-blockchain/_arbitrum.mdx

@@ -10,9 +10,9 @@ import TabItem from '@theme/TabItem'
 <TabItem value="mainnet">
 
 - **Chain ID:** 0xEC0
-- **Public RPC URL:** https://rpc.startale.com/astar-zkevm
+- **Public RPC URL:** `https://rpc.startale.com/astar-zkevm`
 - **Display Name:** Astar zkEVM Mainnet
-- **Block Explorer Link:** https://astar-zkevm.explorer.startale.com
+- **Block Explorer Link:** `https://astar-zkevm.explorer.startale.com`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum
 

embedded-wallets/connect-blockchain/_general-connect-blockchain/_astar-zkevm.mdx

@@ -10,9 +10,9 @@ import TabItem from '@theme/TabItem'
 <TabItem value="testnet">
 
 - **Chain ID:** 0x5C2359
-- **Public RPC URL:** https://rpc.startale.com/astar-zkyoto
+- **Public RPC URL:** `https://rpc.startale.com/astar-zkyoto`
 - **Display Name:** Astar zKyoto Testnet
-- **Block Explorer Link:** https://zkyoto.explorer.startale.com
+- **Block Explorer Link:** `https://zkyoto.explorer.startale.com`
 - **Ticker:** ETH
 - **Ticker Name:** Ethereum