Add /llm.txt path for LLM-friendly documentation access.

[emaadali]

Description

This PR adds the starlight-llms-txt package to docs-starlight to improve how LLMs can navigate the documentation. Specifically, it:

• Introduces a /llm.txt path that is auto-generated from the existing docs-starlight documentation.
• Updates astro.config.mjs to include the new starlightLlms integration.
• Adds llms.ts to handle generation and serving of the LLM-readable content.
• Updates package.json to include the new dependency.

TODOs

Read the Gruntwork contribution guidelines.

• I authored this code entirely myself
• I am submitting code based on open source software (e.g. MIT, MPL-2.0, Apache)
• I am adding or upgrading a dependency or adapted code and confirm it has a compatible open source license
• Update the docs.
• Run the relevant tests successfully, including pre-commit checks.
• Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added starlight-llms-txt to auto-generate a /llm.txt path, giving LLMs access to all documentation content in Markdown format.

Migration Guide

No backward-incompatible changes.

Summary by CodeRabbit

• New Features
  • Adds an integration that exposes organized custom sets of documentation for the docs site, enabling grouped doc collections for downstream use.
• Dependencies
  • Adds a runtime dependency to support the new language-model documentation integration.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[vercel]

@emaadali is attempting to deploy a commit to the Gruntwork Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a new docs-starlight module that converts the site's sidebar into starlight-llms custom sets, registers it as a plugin in the Astro config (invoked with the existing sidebar), and adds the runtime dependency starlight-llms-txt.

Changes

Cohort / File(s)	Summary
Plugin integration and setup docs-starlight/astro.config.mjs	Imports starlightLlms from ./llms and registers it in the Astro integrations/plugins array, calling it with the existing sidebar config.
Sidebar-to-LLM conversion module docs-starlight/llms.ts	New TypeScript module introducing SidebarItem, SidebarSection, Sidebar types, sidebarToCustomSets(sidebar) and a default export. Recursively collects slug-based paths and autogenerated directory patterns, strips numeric prefixes from directory names, deduplicates paths per section, produces { label, paths, description? } custom sets, and calls starlightLlmsTxt with the computed customSets and rawContent flag.
Dependency addition docs-starlight/package.json	Adds dependency starlight-llms-txt with version ^0.6.0.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Inspect recursive traversal and edge cases in docs-starlight/llms.ts (prefix stripping, autogenerated dir handling, deduplication).
• Verify shape of the sidebar passed from astro.config.mjs matches the new module's expected types.
• Confirm correct usage of starlight-llms-txt API and that rawContent/customSets are formed as intended.

Possibly related PRs

• chore: Final pre-migration steps #4688 — Also modifies the Starlight integration in docs-starlight/astro.config.mjs (plugins array changes).
• fix: Adding edit link back #4629 — Related edits to the Starlight integration in docs-starlight/astro.config.mjs (adds editLink handling).

Suggested reviewers

• denis256

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding a /llm.txt path for LLM-friendly documentation access, which directly aligns with the PR's primary objective of introducing the starlight-llms-txt integration.
Description check	✅ Passed	The PR description comprehensively covers the changes, follows the required template structure with Description and TODOs sections, includes release notes and migration guide, and all checklist items are marked complete.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 889c9ca and 03662b4.

📒 Files selected for processing (3)

• docs-starlight/astro.config.mjs (2 hunks)
• docs-starlight/llms.ts (1 hunks)
• docs-starlight/package.json (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

docs-starlight/llms.ts (1)

docs-starlight/astro.config.mjs (2)

• sidebar (18-91)
• sidebar (18-91)

🔇 Additional comments (8)

docs-starlight/astro.config.mjs (2)

13-13: LGTM: Import follows Astro conventions.

The extensionless import from "./llms" follows Astro/Vite module resolution conventions for TypeScript files.

---

208-208: LGTM: Plugin registration is correct.

The plugin is properly initialized with the sidebar configuration and registered in the plugins array.

docs-starlight/llms.ts (5)

3-24: LGTM: Type definitions are well-structured.

The type definitions accurately model the sidebar structure with proper support for nested items and autogeneration patterns.

---

26-30: LGTM: Prefix stripping logic is correct.

The function properly strips numeric prefixes from directory paths using regex, which aligns with the directory naming convention used in the sidebar configuration.

---

32-46: LGTM: Path collection logic is correct.

The recursive function properly traverses the sidebar structure and collects paths from slugs and autogenerated directories. The use of glob patterns (**) for autogenerated directories is appropriate.

---

48-64: LGTM: Sidebar conversion logic is well-implemented.

The function correctly transforms sidebar sections into custom sets with proper path collection and deduplication. The explicit return type enhances type safety.

---

69-72: The rawContent: true configuration is appropriate for LLM training.

Raw Markdown provides cleaner, more structured content for language model consumption compared to compiled output. This setting is the correct choice for generating llms.txt files intended to train LLMs on your documentation.

docs-starlight/package.json (1)

28-28: Verify starlight-llms-txt package version and security status.

The current version ^0.6.0 specification should be verified against the npm registry to confirm it is the latest stable release. No known security vulnerabilities were detected in available sources, but a direct npm registry audit is recommended.

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

docs-starlight/llms.ts (1)

3-24: Consider using types from the library or framework if available.

The manual type definitions work correctly, but if starlight-llms-txt or Starlight exports types for sidebar structures, using those would improve maintainability and reduce drift.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e65029c and 0c41ea1.

📒 Files selected for processing (1)

• docs-starlight/llms.ts (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

docs-starlight/llms.ts (1)

docs-starlight/astro.config.mjs (2)

• sidebar (18-91)
• sidebar (18-91)

🔇 Additional comments (5)

docs-starlight/llms.ts (5)

26-30: LGTM!

The prefix-stripping logic correctly handles multi-segment paths and cleanly removes numeric prefixes like "01-" or "02-" from each segment.

---

32-46: LGTM!

The recursive path collection correctly handles direct slugs, autogenerate directories (with proper prefix stripping), and nested items. The nullish coalescing provides appropriate safety.

---

48-64: LGTM!

The transformation logic is sound. The function correctly aggregates paths from both autogenerate directories and nested items, with appropriate deduplication.

---

66-71: LGTM!

The integration correctly composes the sidebar transformation with the starlightLlmsTxt call. The previous debug logging issue has been resolved.

---

1-1: Library validity confirmed, but version and API options require direct npm registry check.

The starlight-llms-txt library is actively maintained and listed in the official Starlight plugins. However, verify version 0.6.0 is current and that your configuration options match the library's API documentation on npm or the official docs site.

[yhakbar]

Why are we stripping the number prefixes?

[emaadali]

It is just for the starlight-llms-txt package - we don't strip number prefixes for the sidebar config parameter. The starlight-llms-txt package uses the paths to serve the page contents as markdown on-demand. It cannot resolve the path to the docs file if the paths are prefixed with the numbers that are used by the sidebar config parameter for sorting. It has no impact on any of the existing site code.

[yhakbar]

Ah, I think I understand. Are you saying the mismatch between the name of the file and the file slug is what's causing problems here? Should we actually be determining the appropriate slug for the page instead of stripping the leading number prefix? I'm thinking of situations like the logging overview page where the slug is docs/reference/logging, not docs/reference/logging/overview, even though that's what you'd get if you just stripped numbers off the front of the file names.