Model Cost Management: Tracking Spend & Learning When to Upgrade Models

[AmeliaOramous]

Problem

We're running instar agents that call multiple AI models with vastly different costs:

• gpt-4o-mini: ~$0.15/1M input tokens
• gpt-4o: ~$2.50/1M input tokens
• gpt-4x: ~$30/1M input tokens

Without visibility and intelligent selection, costs spiral.

Proposed Solution

A minimal-viable cost management system with three core components:

1. Cost tracking — per-model, per-job cost attribution with daily/weekly reporting
2. Smart tier selection — rule-based heuristics (failures, tool use, context size, code changes) + Phase 2 bandit exploration
3. Budget enforcement — soft caps (warn + suggest downgrade) and hard caps (block)

Key Features (Phase 1)

• Tier defaults: economy (gpt-4o-mini) → standard (gpt-4o) → high (gpt-4x), all tunable
• Auto-upgrade rules: Upgrade from economy when task fails 2+ times, uses 3+ tools, has >4K input tokens, or >500 line code changes
• Budget constraints: Soft caps ($0.50/day, $15/month) warn in logs; hard caps ($1/call, $2/day) block execution
• Per-call instrumentation: Track input/output tokens, model, cost, success/failure for Phase 2 learning
• Daily reporting: Per-model spend breakdown + per-job cost attribution

Phase 2 (Future)

Multi-armed bandit exploration to learn optimal tier per task family:

• 1-in-10 eligible calls explore a higher tier (ε=0.1, capped at 5/day)
• Track success rate + latency for each upgrade trial
• Gradually shift weight toward proven tier combinations

Design Questions for Upstream

1. Integration scope: Should this live in instar core as a scheduler hook, or as an example agent module in docs?
2. Multi-provider support: Interest in provider-agnostic pricing (Claude, OpenAI, etc.)?
3. Reporting targets: Preferred notifications (Telegram, email, or configurable per agent)?
4. Cost ceiling: Does $1 per call hard cap feel right?

Implementation Details

• TypeScript strict mode, zero external dependencies
• Full unit test coverage for cost estimation, heuristics, budget logic
• Pricing map embedded (updated quarterly from public rates)
• RFC at: https://github.com/AmeliaOramous/instar/blob/main/docs/rfcs/2026-03-cost-management.md
• Draft PR incoming for detailed code review

This is a minimal-viable approach that keeps agents economical while preserving capability — agents get the right model when they need it, they just learn when that actually happens.