feat: read shared data files locally instead of fetching from GitHub

[neon60]

Motivation

Every rocm-docs-core build fetched shared data files from GitHub at build time, and this ran in every component repo (ROCm, rocm-systems, rocm-libraries). The version and toolkit files were fetched on the html-page-context event — once per page — so a ~100-page build made roughly 430 raw.githubusercontent.com requests, scaling linearly with page count. In parallel CI, all of these builds drew from the same anonymous GitHub quota simultaneously.

Two of the .txt fetches had no error handling, so a single 429 crashed the build outright; others retried (up to 100×, or looped for up to 10 minutes) which only amplified request volume under throttling. The projects.yaml fetch additionally used the unauthenticated GitHub REST API (60 requests/hour hard limit). Together this produced the rate-limit failures seen across component-repo CI.

Technical Details

• New rocm_docs/common.py — centralizes access to the shared data checkout:
  • get_common_dir() / read_common_file() resolve the checkout location and read data files locally.
  • ensure_common_dir() resolves the checkout during extension setup so a consumer's conf.py needs no configuration. Precedence: rocm_docs_common_dir config value → ROCM_DOCS_COMMON_DIR env var → auto-clone rocm-docs-common to /rocm-docs-common.
  • clone_common_if_missing() clones the checkout when absent and fast-forwards it to the branch tip when already present, so repeated local builds don't read stale data. The refresh is best-effort (a failure logs a warning and uses the existing copy); the initial clone still raises.
  • There is no per-file remote fetch fallback: once resolved, all data files are read from the checkout, and a missing directory or file fails the build.
• rocm_docs/projects.py — _load_projects() reads projects.yaml from the checkout; removed the GitHub-API fetch path, _fetch_projects, MappingFileFetchError, the bundled fallback, and the now-unused os/github/requests imports. Registered the rocm_docs_common_dir config value and calls ensure_common_dir() in setup().- rocm_docs/theme.py — all four data-file fetches now read locally; removed the requests/time/MAX_RETRY retry machinery.
• .readthedocs.yaml — added a post_checkout job that clones rocm-docs-common to the repo root, matching the path ensure_common_dir() resolves to (so the automatic clone is a no-op on RTD). Added the clone directory to .gitignore.
• Docs — added docs/developer_guide/local_common_data.md (and its TOC entry) documenting resolution order, local builds, and RTD behavior.
• Tests — added a tests/common_data/ fixture (placeholder secrets, no real tokens) and an autouse fixture that points test builds at it.
• Prerequisite — none for consumers: builds auto-clone rocm-docs-common by default. To pin a specific checkout (for example, unreleased data), set the rocm_docs_common_dir config value or the ROCM_DOCS_COMMON_DIR env var. The data files previously lived on the new_data/data branches of this repo (and projects.yaml on develop); they now live in rocm-docs-common.

Test Plan

• ruff and mypy run clean on the changed files.
• Full unit test suite via pytest.
• Full ROCm docs build using the ROCM_DOCS_COMMON_DIR environment variable, checking the build log for any remaining rocm-docs-core GitHub fetches.
• Full ROCm docs build using -D rocm_docs_common_dir=... with the env var unset, to exercise the config-value path.
• Build with neither the config value nor env var set, to confirm the automatic clone resolves the checkout.
• Second consecutive local build with the auto-cloned checkout in place, to confirm it refreshes to the branch tip rather than going stale.

Test Result

• ruff: all checks passed. mypy: no issues found.
• pytest: 54 passed.
• Full ROCm build (107 pages) via env var: build succeeded, 0 rocm-docs-core GitHub fetches (down from ~430).
• Full ROCm build via -D rocm_docs_common_dir=... with env var unset: build succeeded, 0 fetches.
• With neither set: build auto-cloned rocm-docs-common and succeeded, 0 data-file fetches.
• Second build with the checkout present: refreshed to the branch tip (git fetch + checkout FETCH_HEAD) before reading data.

Known Issues

• COMMON_REPO_URL in common.py and the post_checkout clone in .readthedocs.yaml currently point at github.com/neon60/rocm-docs-common (a fork). These move to the ROCm-org rocm-docs-common repo before merge.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.