vyos-docs-legacy-snapshot

Static snapshot of the legacy VyOS documentation versions 1.2 and 1.3, taken from the live ReadTheDocs-hosted docs.vyos.io/en/1.2/ and docs.vyos.io/en/1.3/ trees before docs.vyos.io cuts over to Cloudflare Workers (per the 2026-07-02-cloudflare-migration-fresh-spec design, §6 and §15a).

Why this repo exists (one-time nature)

RTD's build pipeline for 1.2 and 1.3 will be retired as part of the docs platform migration. Those two versions are LTS/legacy and no longer receive doc changes, so instead of standing up a live Sphinx build for them on the new platform, we take one static crawl of the RTD-rendered output, scrub out RTD-specific chrome (platform JS, analytics, ad units, the RTD version flyout), inject our own version-picker bundle so the pages stay navigable into the new picker UI, and commit the result as a static asset tree. That tree is served by a small dedicated Worker (vyos-docs-legacy, deployed by Task 4.2) — no crawler, no Sphinx, no CI rebuild. This repo's snapshot/ directory is the source of truth going forward; there is no "re-crawl on a schedule" step.

This is a one-time operation. snapshot/ is not meant to be regenerated routinely — 1.2 and 1.3 are frozen. Re-run tools/snapshot.sh only if:

• The initial snapshot had a defect (missed pages, bad scrub, broken links) that must be corrected from the still-live RTD source, or
• RTD's hosting of 1.2/1.3 changes in a way that requires re-baselining before RTD is decommissioned.

Once RTD stops serving docs.vyos.io/en/1.2/ and /en/1.3/, tools/snapshot.sh can no longer be re-run at all — the source it crawls will be gone. At that point this repo's tools/ become historical record only.

Layout

tools/
  snapshot.sh       # crawl -> scrub -> picker/PDF -> parity -> sitemap -> pagefind
  scrub.py          # HTML scrubbing (RTD chrome removal + URL localization + picker injection)
  test_scrub.py     # TDD unit tests for scrub.py
snapshot/           # committed output: en/1.2/, en/1.3/ (scrubbed HTML, static assets,
                     # per-version pagefind/, per-version sitemap.xml)
r2-staging/         # 1.3's PDF (29.2 MiB, over the 25 MiB Workers per-asset cap) —
                     # uploaded to R2 separately by the Task 4.2 deploy flow, NOT
                     # committed as a Worker static asset. Not committed to this repo
                     # either (large binary; staged locally for the R2 upload step).
crawl/               # scratch — raw wget mirror output, wiped and regenerated by every
                     # snapshot.sh run, NEVER committed
snapshot.new/        # scratch — build-then-swap staging dir; moved onto snapshot/ only
                     # on full success, NEVER committed

Re-running the snapshot

Prerequisites: wget, curl, python3.11+, Node (for npx pagefind), and a local checkout of vyos-documentation (for the version-picker bundle files at docs/_static/{js,css}/version-picker.*).

DOCS_REPO=/path/to/vyos-documentation bash tools/snapshot.sh

DOCS_REPO defaults to $HOME/GitHub/vyos-documentation. The script is idempotent: it wipes crawl/ and snapshot.new/ before every run, crawls /en/1.2/ and /en/1.3/ from the live docs.vyos.io, scrubs every mirrored HTML file via tools/scrub.py, copies in the picker bundle, downloads 1.3's PDF into r2-staging/ (validating its %PDF- header), hard-fails on any dangling internal link (parity check — the crawl graph is the source of truth since 1.2/1.3 never had a sitemap.xml), generates a sitemap.xml per version from the crawl inventory, builds a per-version Pagefind index (no cross-version search contamination), and — only once both 1.2 and 1.3 complete every step successfully — atomically swaps snapshot.new/ onto snapshot/. A failed run therefore never leaves the committed snapshot/ tree partially updated.

Run the unit tests with:

python3.11 -m venv .venv && . .venv/bin/activate && pip install pytest
python -m pytest tools/test_scrub.py -v