Skip to content

fix: count actual tool calls instead of tool-related messages#564

Merged
teknium1 merged 1 commit intoNousResearch:mainfrom
0xbyt4:fix/tool-call-count-accuracy
Mar 10, 2026
Merged

fix: count actual tool calls instead of tool-related messages#564
teknium1 merged 1 commit intoNousResearch:mainfrom
0xbyt4:fix/tool-call-count-accuracy

Conversation

@0xbyt4
Copy link
Contributor

@0xbyt4 0xbyt4 commented Mar 7, 2026

Summary

tool_call_count in the session database was inaccurate due to two bugs in append_message():

Bug 1 — Under-counting parallel tool calls: An assistant message with N parallel tool calls (e.g. "kill the light and shut off the fan" = 2 ha_call_service calls) only incremented tool_call_count by 1 instead of N.

Bug 2 — Over-counting tool responses: Tool response messages (role=tool) also incremented tool_call_count, double-counting every tool interaction.

Combined effect: 2 parallel tool calls produced tool_call_count=3 (1 from assistant + 2 from tool responses) instead of the correct value of 2. This makes /insights and /usage report inflated tool call numbers.

Fix: Only count from assistant messages that carry tool_calls, incrementing by len(tool_calls). Tool response messages no longer affect the counter.

Test plan

  • test_tool_call_count_matches_actual_calls — 2 parallel tool calls + 2 tool responses = count should be 2 (fails on old code with 3)
  • test_tool_response_does_not_increment_tool_count — tool response alone should not increment count
  • test_assistant_tool_calls_increment_by_count — single tool call increments by 1
uv run pytest tests/test_hermes_state.py tests/test_insights.py -q  # 108 passed

tool_call_count was inaccurate in two ways:

1. Under-counting: an assistant message with N parallel tool calls
   (e.g. "kill the light and shut off the fan" = 2 ha_call_service)
   only incremented tool_call_count by 1 instead of N.

2. Over-counting: tool response messages (role=tool) also incremented
   tool_call_count, double-counting every tool interaction.

Combined: 2 parallel tool calls produced tool_call_count=3 (1 from
assistant + 2 from tool responses) instead of the correct value of 2.

Fix: only count from assistant messages with tool_calls, incrementing
by len(tool_calls) to handle parallel calls correctly. Tool response
messages no longer affect tool_call_count.

This impacts /insights and /usage accuracy for sessions with tool use.
@teknium1 teknium1 merged commit 1ec8c1f into NousResearch:main Mar 10, 2026
1 check passed
@teknium1
Copy link
Contributor

Merged to main — thanks @0xbyt4! Good catch on the double-counting from tool responses and under-counting of parallel calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants