Skip to content

Commit 7ebf671

Browse files
authored
Refresh skill validation evidence
Refresh skill validation evidence, model PR histories, open PR watch, B200 validation records, and repeatable update prompt.\n\nValidation:\n- local pytest: 116 passed\n- local pre-commit: passed\n- remote B200 subset: 17 passed, 160 subtests passed\n- remote cookbook validator: passed\n- PR lint: passed
1 parent 9eba646 commit 7ebf671

155 files changed

Lines changed: 114417 additions & 51503 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ intelligence.**
1010
[![GitHub stars](https://img.shields.io/github/stars/BBuf/AI-Infra-Auto-Driven-SKILLS?style=social)](https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/stargazers)
1111
[![GitHub forks](https://img.shields.io/github/forks/BBuf/AI-Infra-Auto-Driven-SKILLS?style=social)](https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/forks)
1212
[![Last commit](https://img.shields.io/github/last-commit/BBuf/AI-Infra-Auto-Driven-SKILLS?style=flat-square)](https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/commits/main)
13-
[![Core skills](https://img.shields.io/badge/core_skills-10-2f80ed?style=flat-square)](#core-skills)
13+
[![Core skills](https://img.shields.io/badge/core_skills-11-2f80ed?style=flat-square)](#core-skills)
1414
[![PR histories](https://img.shields.io/badge/pr_histories-66-2ea44f?style=flat-square)](#model-pr-history-catalog)
1515
[![KDA-Pilot](https://img.shields.io/badge/sibling-KDA--Pilot-ff7b72?style=flat-square)](https://github.com/BBuf/KDA-Pilot)
1616

@@ -43,6 +43,7 @@ find it.
4343
| [`llm-torch-profiler-analysis`](skills/llm-torch-profiler-analysis/) | You need a three-table profiler report that keeps `extend/prefill` and `decode` evidence separate. |
4444
| [`llm-pipeline-analysis`](skills/llm-pipeline-analysis/) | You need forward-pass, layer, and kernel-level timing from a torch profiler trace, including anchor boundaries and Perfetto ranges. |
4545
| [`model-compute-simulation`](skills/model-compute-simulation/) | You need operator shapes, FLOPs, MFU estimates, kernel-to-op mapping, or parallelism what-if analysis for an LLM serving shape. |
46+
| [`model-pr-diff-dossier`](skills/model-optimization/model-pr-diff-dossier/) | You need to create or revise model PR history docs with manual diff-reviewed cards instead of shallow PR-title summaries. |
4647
| [`sglang-humanize-review`](skills/sglang-humanize-review/) | You need SGLang code-review findings grounded in full human PR review episodes from project start through the latest refresh (June 2026), including inline code context, top-level discussion, review summaries, and multi-round replies. Every review opens with a PR comprehension pass — a change summary plus a Mermaid execution flowchart with the diff's modified steps marked — so the reviewer sees how the PR runs before the findings. |
4748
| [`sglang-sota-humanize-loop`](skills/sglang-sota-humanize-loop/) | You want one model-level Humanize RLCR loop that owns SGLang gap decisions against a selected comparison framework set, profiler triage, required layer-pipeline deep dives, SGLang patches, optional `ncu-report-skill` evidence, and real-model revalidation after the fixed fair benchmark. |
4849
| [`vllm-sota-humanize-loop`](skills/vllm-sota-humanize-loop/) | You want one model-level Humanize RLCR loop that owns gap decisions, profiler triage, required layer-pipeline deep dives, vLLM patches, optional `ncu-report-skill` evidence, and real-model revalidation after the fixed fair benchmark. |
@@ -109,6 +110,17 @@ python3 scripts/query.py --framework vllm "qwen3 fused qk norm"
109110
python3 scripts/query.py --framework tokenspeed --model qwen35 qk rmsnorm
110111
```
111112

113+
Open PR freshness is tracked separately from merged history cards:
114+
115+
```bash
116+
python3 tools/check_open_pr_watch.py --format markdown \
117+
--output model-pr-optimization-history/open-pr-watch.md
118+
```
119+
120+
This report uses the GitHub pulls API with an anonymous REST fallback when
121+
`gh api` is rate-limited. If every repo fetch fails, the tool exits non-zero
122+
instead of writing a misleading empty report.
123+
112124
## Evidence Standards
113125

114126
The repo is opinionated about evidence because performance work gets noisy fast.
@@ -136,6 +148,9 @@ The repo is opinionated about evidence because performance work gets noisy fast.
136148
- Model optimization histories should point back to PRs, files, diffs, and risk
137149
surfaces rather than vague summary text; they live as one PR-driven knowledge
138150
base, not per-model skills.
151+
- Root-level [`update_prompt.md`](update_prompt.md) captures the full refresh
152+
and validation workflow for updating this repo again without relying on
153+
memory from a previous run.
139154

140155
## Install
141156

@@ -154,7 +169,7 @@ installed as a single Claude Code plugin via the built-in marketplace flow:
154169
/reload-plugins
155170
```
156171

157-
After reload, the 11 skills appear namespaced as
172+
After reload, the 12 skills appear namespaced as
158173
`ai-infra-auto-driven-skills:<skill-name>` (for example
159174
`ai-infra-auto-driven-skills:sglang-sota-humanize-loop`). Update later with
160175
`/plugin marketplace update ai-infra-auto-driven-skills`.
@@ -175,6 +190,7 @@ ln -s "$PWD/skills/llm-serving-capacity-planner" ~/.claude/skills/llm-serving-ca
175190
ln -s "$PWD/skills/llm-torch-profiler-analysis" ~/.claude/skills/llm-torch-profiler-analysis
176191
ln -s "$PWD/skills/llm-pipeline-analysis" ~/.claude/skills/llm-pipeline-analysis
177192
ln -s "$PWD/skills/model-compute-simulation" ~/.claude/skills/model-compute-simulation
193+
ln -s "$PWD/skills/model-optimization/model-pr-diff-dossier" ~/.claude/skills/model-pr-diff-dossier
178194
ln -s "$PWD/skills/sglang-humanize-review" ~/.claude/skills/sglang-humanize-review
179195
ln -s "$PWD/skills/sglang-sota-humanize-loop" ~/.claude/skills/sglang-sota-humanize-loop
180196
ln -s "$PWD/skills/vllm-sota-humanize-loop" ~/.claude/skills/vllm-sota-humanize-loop
@@ -187,8 +203,8 @@ Restart Claude Code after installing. The skills can then be invoked by name,
187203
for example `[$llm-serving-auto-benchmark]`,
188204
`[$llm-serving-capacity-planner]`, `[$llm-torch-profiler-analysis]`,
189205
`[$llm-pipeline-analysis]`, `[$model-compute-simulation]`,
190-
`[$sglang-humanize-review]`, `[$sglang-sota-humanize-loop]`, or
191-
`[$vllm-sota-humanize-loop]`.
206+
`[$model-pr-diff-dossier]`, `[$sglang-humanize-review]`,
207+
`[$sglang-sota-humanize-loop]`, or `[$vllm-sota-humanize-loop]`.
192208

193209
If you prefer copies instead of symlinks, replace `ln -s` with `cp -R`. Copy
194210
`model-pr-optimization-history` only when you want the agent to query the
@@ -206,6 +222,7 @@ cp -R skills/llm-serving-capacity-planner <agent-skill-dir>/llm-serving-capacity
206222
cp -R skills/llm-torch-profiler-analysis <agent-skill-dir>/llm-torch-profiler-analysis
207223
cp -R skills/llm-pipeline-analysis <agent-skill-dir>/llm-pipeline-analysis
208224
cp -R skills/model-compute-simulation <agent-skill-dir>/model-compute-simulation
225+
cp -R skills/model-optimization/model-pr-diff-dossier <agent-skill-dir>/model-pr-diff-dossier
209226
cp -R skills/sglang-humanize-review <agent-skill-dir>/sglang-humanize-review
210227
cp -R skills/sglang-sota-humanize-loop <agent-skill-dir>/sglang-sota-humanize-loop
211228
cp -R skills/vllm-sota-humanize-loop <agent-skill-dir>/vllm-sota-humanize-loop

model-pr-optimization-history/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ the directory acts as one queryable knowledge base for model-family PR evidence.
1111
- `SKILL.md`: agent instructions for using this directory as knowledge.
1212
- `scripts/query.py`: small local search helper for model slugs, doc paths, and
1313
keyword snippets.
14+
- `open-pr-watch.md`: generated watch list for current open PRs that may affect
15+
benchmark, profiler, and model-history guidance.
1416

1517
Each model history is bilingual when practical (`README.zh.md` and
1618
`README.en.md`) and should be grounded in inspected PR diffs, source files, and
@@ -25,6 +27,10 @@ cutoff and are not yet folded into the timeline / diff-audit cards below. Read
2527
that section first to see what changed most recently before trusting the older
2628
cards.
2729

30+
Open PRs are deliberately kept out of the merged-history cards until their
31+
diffs are manually reviewed. Regenerate `open-pr-watch.md` before long refresh
32+
work and treat it as a triage queue, not as a source of implemented behavior.
33+
2834
Quick queries:
2935

3036
```bash

0 commit comments

Comments
 (0)