Skip to content

fix: extract thoughtsTokenCount for Gemini thinking models#5604

Open
colegottdank wants to merge 1 commit intomainfrom
fix/gemini-thinking-tokens-extraction
Open

fix: extract thoughtsTokenCount for Gemini thinking models#5604
colegottdank wants to merge 1 commit intomainfrom
fix/gemini-thinking-tokens-extraction

Conversation

@colegottdank
Copy link
Collaborator

Summary

  • Add reasoning_tokens extraction from Gemini usageMetadata.thoughtsTokenCount in worker's getDetailedUsage()
  • Add reasoningTokens to GoogleBodyProcessor output in Jawn
  • Fix promptTokens calculation when no caching is present (was returning undefined instead of raw promptTokenCount)

Problem

Gemini thinking models (2.5 Flash, 3.x Pro) return thoughtsTokenCount separately from candidatesTokenCount:

"usageMetadata": {
  "promptTokenCount": 15,
  "candidatesTokenCount": 359,
  "totalTokenCount": 1035,
  "thoughtsTokenCount": 661  // This was not being extracted!
}

This caused:

  1. Token count discrepancies (input + output didn't match total)
  2. Incorrect cost calculations (thinking tokens weren't tracked separately)
  3. Missing reasoning_tokens in the detailed usage breakdown

Test plan

  • Added test for Gemini thinking model response with thoughtsTokenCount
  • Existing GoogleBodyProcessor tests pass
  • Full packages test suite passes (578 tests)

🤖 Generated with Claude Code

- Add reasoning_tokens extraction from Gemini usageMetadata.thoughtsTokenCount
  in worker's getDetailedUsage()
- Add reasoningTokens to GoogleBodyProcessor output in Jawn
- Fix promptTokens calculation when no caching is present
  (was returning undefined instead of raw promptTokenCount)

This fixes incorrect token counting and cost calculations for Gemini
thinking models (2.5 Flash, 3.x Pro) where thoughtsTokenCount was not
being stored separately.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Feb 24, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
helicone Skipped Skipped Feb 24, 2026 7:18am
helicone-bifrost Skipped Skipped Feb 24, 2026 7:18am
helicone-eu Skipped Skipped Feb 24, 2026 7:18am

Request Review

@supabase
Copy link

supabase bot commented Feb 24, 2026

This pull request has been ignored for the connected project bolqqmqbrciybnypvklh because there are no changes detected in supbase/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@supabase
Copy link

supabase bot commented Feb 24, 2026

This pull request has been ignored for the connected project lmahfbbnchpworytrrqk because there are no changes detected in .supabase/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@claude
Copy link
Contributor

claude bot commented Feb 24, 2026

Claude finished @colegottdank's task —— View job


PR Review: Gemini Thinking Tokens Extraction

Reviewing changes for:

  • Code quality and implementation correctness
  • Test coverage and edge cases
  • Performance and security considerations
  • API consistency and breaking changes
  • Provide final assessment and recommendations

Analysis

Changes Overview:

  1. Worker (DBLoggable.ts): Added reasoning_tokens extraction from Gemini usageMetadata.thoughtsTokenCount
  2. Jawn (GoogleBodyProcessor.ts): Added reasoningTokens field and fixed promptTokens calculation
  3. Tests: Added comprehensive test for Gemini thinking model scenario

Detailed Review

Code Quality - Excellent

  • Correct Implementation: Properly extracts thoughtsTokenCount and maps it consistently
  • Important Bug Fix: Fixed critical issue where promptTokens was undefined when no caching present (line 76-78 in googleBodyProcessor.ts)
  • Type Safety: Uses proper optional chaining and null coalescing operators
  • Consistency: Changes align with existing codebase patterns and type definitions

Test Coverage - Comprehensive

  • Realistic Test Data: Uses actual Gemini 2.5 Flash response structure with authentic token counts
  • Proper Assertions: Verifies all token calculations including the new reasoning tokens extraction
  • Edge Case Coverage: Tests the conditional reasoningTokens field behavior (undefined when 0, value when > 0)
  • Regression Prevention: Ensures existing functionality remains intact

Performance & Security - No Issues

  • Minimal Performance Impact: Only adds simple field extraction with no computational overhead
  • No Security Concerns: Changes involve only data parsing/mapping without external dependencies
  • Memory Efficient: Uses existing patterns for optional field handling

API Consistency - Backward Compatible

  • Non-Breaking: All changes are additive (new optional fields)
  • Type Consistency: reasoningTokens field properly typed across all layers (Worker → Jawn → Web)
  • Naming Conventions: Follows existing camelCase patterns (reasoningTokens vs reasoning_tokens)

Score: 9/10 - Excellent implementation, ready to merge

Minor Suggestions:

  1. Documentation: Consider adding a comment explaining the Gemini thinking model token structure for future maintainers
  2. Constants: The conditional logic thoughtsTokenCount > 0 ? thoughtsTokenCount : undefined could use a helper function, but this matches existing patterns in the codebase

Suggestions Summary:

  • No critical fixes needed - Implementation is solid and well-tested
  • No breaking changes - All changes are backward compatible
  • Good test coverage - New test properly validates the thinking token extraction
  • 💡 Optional enhancement: Add inline documentation for the thinking model token logic

Recommendation: Merge with confidence. This PR correctly fixes a significant gap in Gemini thinking model token tracking and includes proper testing.


Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant