Refresh SGLang review corpus to 2026 + backfill model PR histories#68
Merged
Conversation
SGLang humanize-review corpus: - Extend the human review episode corpus from "through 2025-12-31" to the current refresh (project start through 2026), nearly doubling coverage: threads 17,914 -> 32,639, comments 39,511 -> 76,516, with 14,236 threads from 2026 PRs now included. - Fix a truncation bug in the collector: PR conversation comments and review submissions were capped at the first 100 GraphQL nodes per PR, silently dropping the tail of long multi-round discussions. Both connections now paginate fully (max_comments_per_thread 77 -> 142; 8 conversation PRs and 2 review PRs that were previously truncated are now captured in full). - Harden the collector for long runs: read the gh token fresh and refresh it on 401/403, and retry transient connection resets (OSError/HTTPException) with capped backoff. - Rename the corpus to a date-agnostic sglang-review-corpus.jsonl.gz / .metadata.json and use a dynamic window label; update SKILL.md, README, the query helper default, and the tests accordingly. Model PR optimization history: - Prepend a dated "PR Backfill Audit" section to all 62 model-family docs (sglang + vllm). Each audit git-traces the tracked implementation files and lists PR-numbered merges landed after the doc's previous freshness cutoff that are not yet folded into the timeline / diff-audit cards (643 new PRs total). Titles are normalized so the dossier-quality checks stay green. - Document the audit convention in the module README and SKILL. All 105 tests pass; ruff/isort/black clean on the changed scripts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BBuf
marked this pull request as ready for review
June 5, 2026 20:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A timeliness pass over the auto-driven skills, focused on the two areas that
were most stale and most verifiable.
1. SGLang humanize-review corpus → through 2026, full multi-round capture
Extended coverage from project start → 2025-12-31 to project start →
2026 (latest refresh). The corpus roughly doubled:
Fixed a multi-round truncation bug. The collector fetched only the first
100 GraphQL nodes of each PR's
commentsandreviewsconnections, silentlydropping the tail of long discussions. Both now paginate fully via cursor
follow-up queries —
conversation_paginated_prs=8andreview_submission_paginated_prs=2were previously truncated and are nowcaptured end-to-end. This directly addresses the "comments / multi-round
discussion capture may be incomplete" concern.
Hardened the collector for long runs (a full scrape is ~40 min): it now
reads the
ghtoken fresh, refreshes it on401/403(OAuth tokens rotatemid-run), and retries transient connection resets (
OSError/HTTPException)with capped backoff.
Renamed the corpus to a date-agnostic
sglang-review-corpus.jsonl.gz/.metadata.jsonand switched to a dynamic window label; updatedSKILL.md,the repo
README, the query-helper default, and the tests.2. Model PR optimization history → dated backfill audits on all 62 docs
The canonical generator (
tools/rebuild_model_pr_history_from_git.py) needsworktrees + a warm cache + thousands of GitHub API calls to fully regenerate,
which would also rewrite every doc wholesale. Instead, each of the 62
model-family docs (sglang + vllm) gets a conservative, git-traced
## <date> PR Backfill Auditsection prepended after the title. It listsPR-numbered merges that touched that model's tracked implementation files
after the doc's previous freshness cutoff and are not yet folded into the
older timeline / diff-audit cards — 643 new PRs surfaced in total. PR titles
are normalized (agent-tag/emoji stripped) so the dossier-quality test stays
green. The convention is documented in the module README and SKILL.
Intentionally left as-is
The dated GPU/benchmark/profiler/architecture-diagram validation runs in the
other skills are point-in-time evidence (specific hardware + commits + metrics).
I did not invent fresh dates over them, since they can't be honestly "refreshed"
without re-running on hardware.
Validation
pytest tests/→ 105 passed (incl. the humanize-review metadata test nowasserting
source_years == [2024, 2026], and the model-PR dossier-qualitytest across all backfilled docs).
ruff/isort/blackclean on the changed collector and query scripts.🤖 Generated with Claude Code