Skip to content

Refresh SGLang review corpus to 2026 + backfill model PR histories#68

Merged
BBuf merged 2 commits into
mainfrom
codex/sglang-humanize-review-full-discussions
Jun 6, 2026
Merged

Refresh SGLang review corpus to 2026 + backfill model PR histories#68
BBuf merged 2 commits into
mainfrom
codex/sglang-humanize-review-full-discussions

Conversation

@BBuf

@BBuf BBuf commented Jun 4, 2026

Copy link
Copy Markdown
Owner

What

A timeliness pass over the auto-driven skills, focused on the two areas that
were most stale and most verifiable.

1. SGLang humanize-review corpus → through 2026, full multi-round capture

  • Extended coverage from project start → 2025-12-31 to project start →
    2026 (latest refresh)
    . The corpus roughly doubled:

    metric before after
    threads 17,914 32,639
    comments 39,511 76,516
    PR years 2024–2025 2024–2026 (14,236 threads from 2026 PRs)
    max comments / thread 77 142
  • Fixed a multi-round truncation bug. The collector fetched only the first
    100 GraphQL nodes of each PR's comments and reviews connections, silently
    dropping the tail of long discussions. Both now paginate fully via cursor
    follow-up queries — conversation_paginated_prs=8 and
    review_submission_paginated_prs=2 were previously truncated and are now
    captured end-to-end. This directly addresses the "comments / multi-round
    discussion capture may be incomplete" concern.

  • Hardened the collector for long runs (a full scrape is ~40 min): it now
    reads the gh token fresh, refreshes it on 401/403 (OAuth tokens rotate
    mid-run), and retries transient connection resets (OSError/HTTPException)
    with capped backoff.

  • Renamed the corpus to a date-agnostic sglang-review-corpus.jsonl.gz /
    .metadata.json and switched to a dynamic window label; updated SKILL.md,
    the repo README, the query-helper default, and the tests.

2. Model PR optimization history → dated backfill audits on all 62 docs

The canonical generator (tools/rebuild_model_pr_history_from_git.py) needs
worktrees + a warm cache + thousands of GitHub API calls to fully regenerate,
which would also rewrite every doc wholesale. Instead, each of the 62
model-family docs (sglang + vllm) gets a conservative, git-traced
## <date> PR Backfill Audit
section prepended after the title. It lists
PR-numbered merges that touched that model's tracked implementation files
after the doc's previous freshness cutoff and are not yet folded into the
older timeline / diff-audit cards — 643 new PRs surfaced in total. PR titles
are normalized (agent-tag/emoji stripped) so the dossier-quality test stays
green. The convention is documented in the module README and SKILL.

Intentionally left as-is

The dated GPU/benchmark/profiler/architecture-diagram validation runs in the
other skills are point-in-time evidence (specific hardware + commits + metrics).
I did not invent fresh dates over them, since they can't be honestly "refreshed"
without re-running on hardware.

Validation

  • pytest tests/105 passed (incl. the humanize-review metadata test now
    asserting source_years == [2024, 2026], and the model-PR dossier-quality
    test across all backfilled docs).
  • ruff / isort / black clean on the changed collector and query scripts.

🤖 Generated with Claude Code

BBuf and others added 2 commits June 4, 2026 19:32
SGLang humanize-review corpus:
- Extend the human review episode corpus from "through 2025-12-31" to the
  current refresh (project start through 2026), nearly doubling coverage:
  threads 17,914 -> 32,639, comments 39,511 -> 76,516, with 14,236 threads
  from 2026 PRs now included.
- Fix a truncation bug in the collector: PR conversation comments and review
  submissions were capped at the first 100 GraphQL nodes per PR, silently
  dropping the tail of long multi-round discussions. Both connections now
  paginate fully (max_comments_per_thread 77 -> 142; 8 conversation PRs and
  2 review PRs that were previously truncated are now captured in full).
- Harden the collector for long runs: read the gh token fresh and refresh it
  on 401/403, and retry transient connection resets (OSError/HTTPException)
  with capped backoff.
- Rename the corpus to a date-agnostic sglang-review-corpus.jsonl.gz /
  .metadata.json and use a dynamic window label; update SKILL.md, README,
  the query helper default, and the tests accordingly.

Model PR optimization history:
- Prepend a dated "PR Backfill Audit" section to all 62 model-family docs
  (sglang + vllm). Each audit git-traces the tracked implementation files and
  lists PR-numbered merges landed after the doc's previous freshness cutoff
  that are not yet folded into the timeline / diff-audit cards (643 new PRs
  total). Titles are normalized so the dossier-quality checks stay green.
- Document the audit convention in the module README and SKILL.

All 105 tests pass; ruff/isort/black clean on the changed scripts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@BBuf
BBuf marked this pull request as ready for review June 5, 2026 20:00
@BBuf BBuf changed the title [codex] Expand SGLang human review corpus episodes Refresh SGLang review corpus to 2026 + backfill model PR histories Jun 5, 2026
@BBuf
BBuf merged commit 1e9d84b into main Jun 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant