Built-in UsageTracker for session-level usage aggregation and cost estimation

[debu-sinha]

This proposes a built-in UsageTracker class that plugs into the existing hooks system to provide session-level token usage aggregation and cost estimation across providers.

Problem

Instructor has a great hooks system and per-call retry-level usage tracking, but there's no built-in way to:

1. Aggregate usage across multiple calls in a session
2. Normalize token counts across providers (OpenAI, Anthropic, Bedrock, Gemini)
3. Estimate costs without writing custom handler code

Every user who wants usage visibility today has to write their own handler from scratch (as shown in examples/hooks/run.py). The existing instructor usage CLI command only works with OpenAI's API endpoint, so users of other providers have no usage tracking at all.

Proposed Solution

A UsageTracker that activates with one line and uses the existing completion:kwargs, completion:response, and completion:error hooks:

import instructor
from instructor import UsageTracker

client = instructor.from_openai(openai.OpenAI())
tracker = UsageTracker.attach(client)

# ... make calls ...

print(tracker.total_tokens)     # 4521
print(tracker.total_cost)       # $0.0234
print(tracker.summary())        # Pretty-printed breakdown by model
print(tracker.to_dict())        # Full data for logging/export

Key design points:

• No new hooks needed -- uses the existing 3 hooks
• Provider-agnostic -- normalizes OpenAI, Anthropic, Bedrock, and Gemini usage into a unified UsageRecord
• Built-in cost estimation with extensible pricing via CostCalculator
• Thread-safe for async and concurrent usage
• Zero dependencies beyond what instructor already has
• Per-call and aggregate views -- individual records + running totals

Design Doc

Full design with data model, provider normalization table, alternatives considered, and testing plan: https://github.com/debu-sinha/design-docs/blob/main/instructor/usage-tracker-design-doc.md

Scope

• instructor/usage_tracker.py -- UsageTracker, UsageRecord, CostCalculator
• instructor/__init__.py -- export UsageTracker
• tests/test_usage_tracker.py -- unit tests (no API keys)
• examples/hooks/usage_tracking.py -- example

No changes to hooks.py, retry.py, or client.py.

Happy to implement this if the direction looks right. A couple open questions in the design doc worth discussing (e.g., whether attach() should be a method on Instructor itself, whether to track latency).

[debu-sinha]

Closing this -- after digging deeper into the hooks architecture, the design has concurrency issues that would need a different approach (no request context to correlate on_request/on_response in async code, and completion:response fires per-attempt not per-call which breaks session-level tracking). The existing hooks + examples/hooks/run.py already cover the common case well enough, and tools like LiteLLM and Langfuse handle cost tracking at a better layer.