Skip to content

docs: propose verifier-bound skill scorecards - #570

Merged
solomonneas merged 1 commit into
mainfrom
t3code/plan-skill-scorecards
Jul 26, 2026
Merged

docs: propose verifier-bound skill scorecards#570
solomonneas merged 1 commit into
mainfrom
t3code/plan-skill-scorecards

Conversation

@solomonneas

Copy link
Copy Markdown
Collaborator

Summary

  • Defines multi-dimension skill scorecards whose eligibility and values come only from verifier receipt v2 fields.
  • Adds a sharp subject, patch, work-owner, verifier, check-role, and failure-taxonomy trust boundary.
  • Specifies bounded exploration, dual effectiveness and utility promotion, immediate routing demotion, strict cold start, and audit-only backfill.
  • Decomposes the implementation into four new S/M issues plus existing outcome ledger: dual-criterion promotion (effectiveness AND utility retention) #503.

Current failure mode

scored_artifact_count is only the number of distinct artifact ids with parsed outcome rows. Capture is opt-in, rank is read-only, and only explicit brigade outcome reconcile --apply advances promotion state. The proposal does not treat the reported 1,601 existing rows as trustworthy training data.

Verification

  • brigade work verify run --target . --command "git diff --cached --check" --command "brigade handoff lint --target . .claude/memory-handoffs/2026-07-26-skill-scorecards-planning.md" --capture brigade-work
  • Receipt: 20260726-223449-work-verify-3713cb
  • MiseLedger evidence: 68adffeab4dd404c3b95b47d
  • Both commands exited 0.

Refs #502 and #503.

Co-Authored-By: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4221d161-dd60-4e8c-ad23-24a6befc73fc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch t3code/plan-skill-scorecards

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@solomonneas

Copy link
Copy Markdown
Collaborator Author

Blocking readiness on one stale dependency reference:

I found no other issue in the scorecard decomposition or its four filed implementation slices.

@solomonneas
solomonneas marked this pull request as ready for review July 26, 2026 22:49
@solomonneas
solomonneas merged commit 495607b into main Jul 26, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant