🤖 Part of #5005.
Description
Add PPL/gap eval sets for diffs, patches, and repo history. Raw source files do not cover the syntax and distribution agents produce when editing code.
Initial sources:
Break out slices for raw git diff, commit-message-plus-diff, PR-review-plus-diff, and issue-to-patch contexts.
Definition of Done
- Add raw eval builders for at least two diff/patch sources.
- Mask provenance fields that should not be predicted.
- Produce separate metrics for patch text versus surrounding issue/review text.
- Add docs for train/eval leakage checks.
🤖 Part of #5005.
Description
Add PPL/gap eval sets for diffs, patches, and repo history. Raw source files do not cover the syntax and distribution agents produce when editing code.
Initial sources:
Break out slices for raw
git diff, commit-message-plus-diff, PR-review-plus-diff, and issue-to-patch contexts.Definition of Done