Skip to content

Commit 8dbd727

Browse files
CopilotJReinhold
andcommitted
fix: return mcpToolsSummary when expectedToolCount is set even if totalCalls is 0
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
1 parent 67fc5d6 commit 8dbd727

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

eval/lib/graders/mcp-tools.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -283,8 +283,9 @@ export async function gradeMcpTools(trialArgs: TrialArgs): Promise<McpToolsSumma
283283
taskConfig.expectedMcpTools,
284284
);
285285

286-
// Only return if there were any MCP tool calls
287-
if (mcpToolsSummary.totalCalls > 0) {
286+
// Return if there were any MCP tool calls, or if expectations were configured
287+
// (so a run that ignores MCP entirely scores 0 rather than undefined)
288+
if (mcpToolsSummary.totalCalls > 0 || mcpToolsSummary.expectedToolCount) {
288289
return mcpToolsSummary;
289290
}
290291

0 commit comments

Comments
 (0)