Add robots.txt and sitemaps reference page

[kathayl]

Summary

Screenshots (optional)

Documentation checklist

• Is there a changelog entry (guidelines)? If you don't add one for something awesome and new (however small) — how will our customers find out? Changelogs are automatically posted to RSS feeds, the Discord, and X.
• The change adheres to the documentation style guide.
• If a larger change - such as adding a new page- an issue has been opened in relation to any incorrect or out of date information that this PR fixes.
• Files which have changed name or location have been allocated redirects.

[github-actions]

This pull request requires reviews from CODEOWNERS as it changes files that match the following patterns:

Pattern	Owners
/src/content/docs/browser-rendering/	@mchenco, @cloudflare/pcx-technical-writing, @celso, @kathayl, @ToriLindsay
*	@cloudflare/pcx-technical-writing

[github-actions]

Preview URL: https://75e1b945.preview.developers.cloudflare.com
Preview Branch URL: https://br-robots-txt-reference.preview.developers.cloudflare.com

Files with changes (up to 15)

Original Link	Updated Link
https://developers.cloudflare.com/browser-rendering/reference/robots-txt/	https://br-robots-txt-reference.preview.developers.cloudflare.com/browser-rendering/reference/robots-txt/

[github-actions]

CI run failed: build logs