Skip to content

agent platform: add pass@k curve to model evaluation metric summaries - #1232

Open
potturu-vamsi wants to merge 1 commit into
digitalocean:mainfrom
potturu-vamsi:vamsi/genai-pass-at-k-curve
Open

potturu-vamsi wants to merge 1 commit into
digitalocean:mainfrom
potturu-vamsi:vamsi/genai-pass-at-k-curve

Conversation

@potturu-vamsi

Copy link
Copy Markdown
Contributor

Adds PassAtKCurve to MetricResultSummary and a new PassAtKPoint type
(K, PassAtKPercent), matching the pass_at_k_curve field the API now returns
for multi-epoch model evaluation runs.

The curve gives pass@k at each k from 1 up to epochs, per metric. It is optional
and only set when epochs > 1, so existing responses decode unchanged.

Matches the OpenAPI spec change in digitalocean/openapi (link the openapi PR here).

Tested: go test . passes; TestGetModelEvaluationRun now asserts the curve decodes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant