feat: serve markdown to agents via Accept header negotiation

[devin-ai-integration]

Summary

Agents asking for Accept: text/markdown got HTML back, even though @signalwire/docusaurus-plugin-llms-txt already emits a markdown twin for every page (/docs/reference/okteto-manifest/ → /docs/reference/okteto-manifest.md). This adds a Netlify Edge Function that serves the twin when markdown is explicitly preferred, leaving HTML as the default for browsers.

Cloudflare's Markdown for Agents doesn't apply — the site is served by Netlify — and Netlify [[redirects]] can't branch on a request header, so the negotiation lives in netlify/edge-functions/markdown-negotiation.js:

// /docs and /docs/*, excluding /docs/assets and any dotted last segment
if (!acceptsMarkdown(accept)) return withVaryAccept(context.next());  // browsers
const md = await fetch(markdownTwin(pathname));                       // /docs/x/ -> /docs/x.md
if (!md.ok) return withVaryAccept(context.next());                    // no twin -> HTML
// Content-Type: text/markdown; charset=utf-8, Vary: Accept, x-markdown-tokens

Details worth knowing:

• acceptsMarkdown compares q-values: text/markdown must be listed explicitly and rank at least as high as text/html, application/xhtml+xml, or */*, so a browser's text/html,...,*/*;q=0.8 keeps getting HTML.
• Any missing twin or fetch failure falls through to the normal HTML response — negotiation can never break a page. Versioned paths (/docs/1.47/...) hit this path by design, since includeVersionedDocs: false means they have no twins.
• Vary: Accept is set on both branches so caches don't cross-serve HTML and markdown.
• x-markdown-tokens is a deterministic length / 4 estimate (no tokenizer dependency at the edge), and is omitted for HEAD where there is no body.
• Upstream Content-Length / Content-Encoding / Transfer-Encoding are stripped because the body is re-serialized, and excludedPattern uses RE2-safe regexes (Netlify compiles these with Go's regexp — no lookaheads).

Verified locally against the built site with the Netlify edge runtime: markdown Accept on /docs/ and /docs/reference/okteto-manifest/ returns text/markdown + x-markdown-tokens, browser-like Accept returns text/html, and a versioned path stays HTML. yarn build passes.

Link to Devin session: https://app.devin.ai/sessions/7151c6c899e64895a9eec5f436cc7354
Requested by: @rberrelleza

[devin-ai-integration]

Original prompt from Ramiro

@okteto/docs Goal: Return HTML responses as markdown when agents request it

Issue: Site does not support Markdown for Agents

Fix: Enable Markdown for Agents so requests with Accept: text/markdown return a markdown version of your HTML response while HTML stays the default for browsers. Confirm the response uses Content-Type: text/markdown (and x-markdown-tokens if available).

Skill: https://isitagentready.com/.well-known/agent-skills/markdown-negotiation/SKILL.md

Docs: https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[netlify]

✅ Deploy Preview for okteto-docs ready!

Name	Link
🔨 Latest commit	a71efdb
🔍 Latest deploy log	https://app.netlify.com/projects/okteto-docs/deploys/6a8c868d7d9bf10008204bdf
😎 Deploy Preview	https://deploy-preview-1332--okteto-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.