Repository navigation
chore: deep review cleanup — drop eval-run artifacts, sync docs, fix stale paths - #2
Merged
Merged
Conversation
…stale paths - remove committed eval-run outputs (security-audit-workspace/, efficiency-audit-workspace/, _fixtures/run_map.json) and gitignore the pattern; keep the vuln-app fixture the evals reference - evals.json (both audits): machine-specific /home/bguest absolute fixture paths -> repo-relative skills/_fixtures/vuln-app - README + skills.sh.json: add the missing security-audit and efficiency-audit skills (table, layout, new Audits/Debugging groupings) - IDEAS.md: fix stale claude-memory/ locations for fast-hypothesis and reproduce-bug (they live in this repo) - assign-mr-reviewers: reference the script relative to the skill dir instead of a hardcoded ~/.claude path (portable across agents); report no_eligible_candidates instead of already_has_reviewers when reviewers are needed but everyone is excluded - fast-hypothesis / reproduce-bug: add "Use when" trigger phrasing to descriptions, matching the rest of the repo Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017WKCdwpi8LBorsiztvhZss
There was a problem hiding this comment.
Pull request overview
This PR cleans up committed audit/eval run artifacts, makes the audit eval fixtures portable by switching from machine-specific absolute paths to repo-relative paths, and synchronizes documentation/groupings so the new audit and debugging skills show up consistently across the repo.
Changes:
- Removed generated eval-run outputs/benchmarks from the repo and added
.gitignorepatterns to prevent re-committing them. - Updated
security-audit/efficiency-auditeval fixtures and prompts to use repo-relative fixture paths. - Updated docs and skill metadata (README, skills.sh.json, IDEAS, SKILL.md descriptions) and made
assign-mr-reviewersscript path + status reporting more portable/precise.
Reviewed changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| skills/security-audit/evals/evals.json | Switch eval fixture/prompt paths to repo-relative fixture location. |
| skills/efficiency-audit/evals/evals.json | Switch eval fixture/prompt paths to repo-relative fixture location. |
| skills/reproduce-bug/SKILL.md | Expand description with consistent “Use when …” trigger phrasing. |
| skills/fast-hypothesis/SKILL.md | Expand description with consistent “Use when …” trigger phrasing. |
| skills/assign-mr-reviewers/SKILL.md | Document portable script path resolution and new status meaning. |
| skills/assign-mr-reviewers/scripts/assign_mr_reviewers.py | Refine “no picks” status to distinguish “already has reviewers” vs “no eligible candidates”. |
| skills.sh.json | Add new “Audits” and “Debugging” groupings including the new skills. |
| README.md | Add missing skills to the table and document their locations in the repo layout section. |
| IDEAS.md | Fix stale skill locations to point at this repo’s skills/ paths. |
| .gitignore | Ignore generated skills/*-workspace/ outputs and skills/_fixtures/run_map.json. |
| skills/_fixtures/run_map.json | Remove committed run-map artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/without_skill/timing.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/without_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/without_skill/grading.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/with_skill/timing.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/with_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/with_skill/grading.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-1/eval_metadata.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/without_skill/timing.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/without_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/without_skill/grading.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/with_skill/timing.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/with_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/with_skill/grading.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/eval-0/eval_metadata.json | Remove committed eval-run artifact. |
| skills/security-audit-workspace/iteration-1/benchmark.md | Remove committed benchmark artifact. |
| skills/security-audit-workspace/iteration-1/benchmark.json | Remove committed benchmark artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/without_skill/timing.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/without_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/without_skill/grading.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/with_skill/timing.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/with_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/with_skill/grading.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-1/eval_metadata.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/without_skill/timing.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/without_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/without_skill/grading.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/with_skill/timing.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/with_skill/outputs/report.md | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/with_skill/grading.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/eval-0/eval_metadata.json | Remove committed eval-run artifact. |
| skills/efficiency-audit-workspace/iteration-1/benchmark.md | Remove committed benchmark artifact. |
| skills/efficiency-audit-workspace/iteration-1/benchmark.json | Remove committed benchmark artifact. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
275
to
+279
| if not picks: | ||
| output["status"] = "already_has_reviewers" | ||
| needed = required - len(details["reviewers"]) | ||
| output["status"] = ( | ||
| "already_has_reviewers" if needed <= 0 else "no_eligible_candidates" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
efficiency-audit-workspace/, _fixtures/run_map.json) and gitignore the
pattern; keep the vuln-app fixture the evals reference
paths -> repo-relative skills/_fixtures/vuln-app
efficiency-audit skills (table, layout, new Audits/Debugging groupings)
reproduce-bug (they live in this repo)
instead of a hardcoded ~/.claude path (portable across agents); report
no_eligible_candidates instead of already_has_reviewers when reviewers
are needed but everyone is excluded
descriptions, matching the rest of the repo
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_017WKCdwpi8LBorsiztvhZss