Release: MRVPP ATBD v2 and PUM v1 #67
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # CLMS Technical Library — local PDF preview font setup. | ||
| # | ||
| # Makes the bundled Typst fonts (_meta/theme/typst-fonts/) discoverable so that | ||
| # rendering a DOCS/*.qmd to PDF — e.g. via the RStudio "Render" button — uses the | ||
| # CLMS fonts (Lato, Liberation Sans, JetBrains Mono) without installing anything | ||
| # system-wide. Quarto's typst format has no font-path option, so we set Typst's | ||
| # native TYPST_FONT_PATHS env var here; RStudio runs this file on session start, | ||
| # and the quarto render it launches inherits the variable. | ||
| # | ||
| # Preview only — the CI build (build-docs.sh) does not use this. | ||
| local({ | ||
| font_dir <- normalizePath( | ||
| file.path(getwd(), "_meta", "theme", "typst-fonts"), | ||
| mustWork = FALSE | ||
| ) | ||
| if (dir.exists(font_dir)) { | ||
| existing <- Sys.getenv("TYPST_FONT_PATHS") | ||
| paths <- if (nzchar(existing)) { | ||
| strsplit(existing, .Platform$path.sep, fixed = TRUE)[[1]] | ||
| } else { | ||
| character(0) | ||
| } | ||
| if (!(font_dir %in% paths)) { | ||
| Sys.setenv( | ||
| TYPST_FONT_PATHS = paste(c(font_dir, paths), collapse = .Platform$path.sep) | ||
| ) | ||
| message("CLMS: TYPST_FONT_PATHS set to bundled fonts for local PDF preview.") | ||
| } | ||
| } | ||
| }) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| #!/usr/bin/env python3 | ||
| """Strip residual block-level HTML wrappers from generated .llms.md sidecars. | ||
|
|
||
| The gfm writer + the simplify_tables_gfm.lua filter already linearize tables and | ||
| remove styling, but Quarto injects crossref float wrappers (`<div id="tbl-…">` … | ||
| `</div>`) and the occasional layout `<div style="overflow-x:…">` AFTER the pandoc | ||
| filters run, so they can't be removed at filter level. These bare wrapper lines are | ||
| pure noise in the plain-text companion the RAG ingests. This post-render pass removes | ||
| standalone block-level `<div>`/`</div>` lines (inline tags like <sub>/<sup> in prose | ||
| are left intact). | ||
|
|
||
| Idempotent. Usage: | ||
| python3 clean_llms_md.py _site # walk *.llms.md under a directory | ||
| python3 clean_llms_md.py a.llms.md b.md # specific files | ||
| """ | ||
|
|
||
| import re | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| # A line that is ONLY an opening/closing <div …> (optionally indented). Block-level | ||
| # wrapper noise — never matches inline tags mid-prose. | ||
| _BARE_DIV = re.compile(r"^[ \t]*</?div\b[^>]*>[ \t]*$") | ||
|
|
||
|
|
||
| def clean_text(text: str) -> str: | ||
| out = [ln for ln in text.split("\n") if not _BARE_DIV.match(ln)] | ||
| cleaned = "\n".join(out) | ||
| # collapse the blank-line runs a removed wrapper can leave behind (3+ → 2) | ||
| cleaned = re.sub(r"\n{3,}", "\n\n", cleaned) | ||
| return cleaned | ||
|
|
||
|
|
||
| def clean_file(path: Path) -> bool: | ||
| original = path.read_text(encoding="utf-8") | ||
| cleaned = clean_text(original) | ||
| if cleaned != original: | ||
| path.write_text(cleaned, encoding="utf-8") | ||
| return True | ||
| return False | ||
|
|
||
|
|
||
| def _iter_targets(args): | ||
| for a in args: | ||
| p = Path(a) | ||
| if p.is_dir(): | ||
| yield from p.rglob("*.llms.md") | ||
| elif p.exists(): | ||
| yield p | ||
|
|
||
|
|
||
| def main(argv) -> int: | ||
| if not argv: | ||
| print(__doc__) | ||
| return 1 | ||
| changed = 0 | ||
| for path in _iter_targets(argv): | ||
| if clean_file(path): | ||
| changed += 1 | ||
| print(f"clean_llms_md: cleaned {changed} file(s)") | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main(sys.argv[1:])) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,17 +6,22 @@ | |
| version, so we fill it each build, like intros and changelogs. | ||
|
|
||
| The value is the tracked version from .llm_cache/versions.json, looked up via the | ||
| original-filename field. The doc's own `version:` frontmatter is never trusted as | ||
| input - it is output we overwrite. No record yet -> seed one (max(major,1).0.0, | ||
| so a _v0 or a name without _vN starts at 1.0.0) so the doc becomes tracked. No | ||
| bump is computed here. | ||
| """ | ||
|
|
||
| import argparse | ||
| import json | ||
| import re | ||
| import sys | ||
| from datetime import date | ||
| from pathlib import Path | ||
|
|
||
| sys.path.insert(0, str(Path(__file__).resolve().parents[1])) # .github/scripts | ||
| from helpers.json_io import load_json_or_empty | ||
| from helpers.doc_types import element_off | ||
|
|
||
| VERSIONS_FILE = ".llm_cache/versions.json" | ||
|
|
||
|
|
@@ -68,7 +73,8 @@ def main(): | |
| vf = Path(args.versions_file) if args.versions_file else repo_root / VERSIONS_FILE | ||
| versions = load_json_or_empty(vf, label="versions") | ||
|
|
||
| filled = baseline = seeded = 0 | ||
| today = date.today().isoformat() | ||
| for qmd in sorted(Path(args.docs_dir).rglob("*.qmd")): | ||
| if {"_site", ".quarto", "_meta"} & set(qmd.parts): | ||
| continue | ||
|
|
@@ -78,32 +84,54 @@ def main(): | |
| continue | ||
| _, end = bounds | ||
|
|
||
| # version-off types keep their own `version:` header, if any; don't touch. | ||
| if element_off(fm_value(lines, end, "type"), "version"): | ||
| continue | ||
|
|
||
| src = fm_value(lines, end, "original-filename") | ||
| mj = major_from_name(src) if src else None | ||
| if mj is None: | ||
| mj = major_from_name(qmd.name) | ||
| # First published version is 1.0.0: a _v0 or a name without _vN starts | ||
| # at 1.0.0, while _v4 etc. keep their filename major. | ||
| baseline_major = max(mj, 1) if mj is not None else 1 | ||
|
|
||
| key = f"DOCS/{src}" if src else None | ||
| tracked = None | ||
| if key: | ||
| entry = versions.get(key) or versions.get(src) | ||
| tracked = (entry or {}).get("current_version") | ||
|
|
||
| if tracked and (mj is None or tracked.split(".")[0] == str(mj)): | ||
|
There was a problem hiding this comment.
For a AGENTS.md reference: AGENTS.md:L92-L93 Useful? React with 👍 / 👎. |
||
| version = tracked | ||
| else: | ||
| version = f"{baseline_major}.0.0" | ||
| baseline += 1 | ||
| # No record yet: seed one so the doc becomes tracked, rather than | ||
| # re-deriving the fallback every build. Keyed by original-filename. | ||
| if key and key not in versions: | ||
| versions[key] = { | ||
| "current_version": version, | ||
| "last_bump": "initial", | ||
| "last_bump_reason": "First release", | ||
| "last_release_tag": "initial", | ||
| "last_updated": today, | ||
| "major_from_filename": baseline_major, | ||
| } | ||
| seeded += 1 | ||
|
|
||
| if set_version(lines, end, version): | ||
| qmd.write_text("".join(lines), encoding="utf-8") | ||
| filled += 1 | ||
|
|
||
| if seeded: | ||
| vf.parent.mkdir(parents=True, exist_ok=True) | ||
| with vf.open("w", encoding="utf-8") as f: | ||
| json.dump(versions, f, indent=2, sort_keys=True) | ||
|
|
||
| print( | ||
| f"[fill_version] set version on {filled} files " | ||
| f"({baseline} used max(major,1).0.0 baseline, {seeded} new cache entries)" | ||
| ) | ||
|
|
||
|
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The newly added
clean_llms_md.pydocuments that Quarto injects residual<div>wrappers after Pandoc filters run, but the inspectedbuild-docs.shpost-render sequence only strips disabled sidecars and never invokes that cleaner. Consequently every deployed.llms.mdcompanion still contains the wrapper noise that the new script was introduced to remove before RAG ingestion; call it on_sitebefore generating the LLM sitemap.Useful? React with 👍 / 👎.