Support markdown for docs and guides

[aliabd]

Was slightly more complicated than i anticipated.

This adds support for crawling the website in markdown! The way it works is that it intercepts the request and checks if the user agent is an llm crawler. If it is then:

• for the guides we just return the content (which is written in markdown anyway)
• for the docs, we use the svx to markdown converter which relies on the AST from mdsvex as well as a ton of regex to parse the content. Does things like converting the paramtable svelte to a markdown table (kept having issues where the | pipe in the types was breaking the table lol). I mostly used claude here to get it to work correctly.

Also added a copy page button and open in huggingchat to the docs pages just like the one we had in guides. You can use it to quickly test if the markdown is correct for the page.

Will continue to test this and make sure everything looks good

Closes: #12586

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/76f9bdc411e82def1da0ca8f0db39998e0be2103/gradio-6.9.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@76f9bdc411e82def1da0ca8f0db39998e0be2103#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/76f9bdc411e82def1da0ca8f0db39998e0be2103/gradio-client-2.1.0.tgz

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
website	minor

---

• Support markdown for docs and guides

---

✅ Changeset approved by @aliabd

• Maintainers can remove approval by unchecking this checkbox.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[freddyaboulton]

Thanks @aliabd ! This is going to be super useful. I'm getting empty markdown for both Interface and gr.DateTime components though. I made sure I was on the main branch of the docs:

[aliabd]

@freddyaboulton i made a bunch of fixes and you shouldn't be seeing that issue anymore. Also figured out how to do content negotiation using cloudflare functions, while keeping the site static and now everything is working as expected!

here's an example with claude code:

gradio.app

preview website

to test out what the markdown for a page looks like easiest way is just the copy button. but to test out the content negotiation easiest way is curl

curl -H "Accept: text/markdown" https://ca4021be.gradio-website.pages.dev/docs/gradio/audio

[freddyaboulton]

This is awesome @aliabd ! The conversion works seamlessly but I found some issues with the conversion function:

Tips are rendered as html

<gradio-app> tags are included in the markdown

`<class 'inspect._empty'> in chatbot docs.

Weird type present in chatbot markdown docs but it may be an issue with docs generation itself because I see the same thing in html actually

[freddyaboulton]

I would say the "" issue is the only blocker here