Skip to content

Commit 134859c

Browse files
committed
docs updated
1 parent fcf67c2 commit 134859c

3 files changed

Lines changed: 84 additions & 0 deletions

File tree

docs/changelog.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,36 @@ The Changelog documents all notable changes made to FastroAI. This includes new
66

77
---
88

9+
## [0.4.0] - Dec 19, 2025
10+
11+
#### Added
12+
- **Enhanced Cost Tracking** by [@igorbenav](https://github.com/igorbenav)
13+
- Cache token tracking (`cache_read_tokens`, `cache_write_tokens`) for accurate cost calculation with prompt caching
14+
- Audio token tracking for multimodal models
15+
- Request count tracking (`request_count`) for API call monitoring
16+
- Tool call count tracking (`tool_call_count`) for agentic behavior metrics
17+
- Provider-specific usage details (`usage_details` dict) for reasoning tokens, etc.
18+
19+
- **Accurate Prompt Caching Costs**
20+
- Cached tokens are now priced at 90% discount (Anthropic) automatically
21+
- `CostCalculator.calculate_cost()` accepts optional `cache_read_tokens`, `cache_write_tokens` parameters
22+
- Pricing overrides support cache token rates via `add_pricing_override(cache_read_per_mtok=..., cache_write_per_mtok=...)`
23+
- Fixes cost overreporting when prompt caching is enabled (~18% more accurate)
24+
25+
#### Changed
26+
- All Pydantic schemas now use `Field(description=...)` for better API documentation
27+
- `ChatResponse`, `StepUsage`, and `PipelineUsage` include new usage fields with sensible defaults
28+
- Backward compatible - existing code continues to work without changes
29+
30+
#### Documentation
31+
- Updated FastroAgent guide with new response fields
32+
- Added Prompt Caching section to Cost Calculator guide
33+
- Updated API reference with enhanced schemas
34+
35+
**Full Changelog**: https://github.com/benavlabs/fastroai/compare/v0.3.0...v0.4.0
36+
37+
---
38+
939
## [0.3.0] - Dec 17, 2025
1040

1141
#### Added

docs/guides/cost-calculator.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,27 @@ print(f"Session cost: ${calc.microcents_to_dollars(total_cost):.4f}")
4747

4848
After 10,000 additions, you have an exact count. No cumulative rounding errors.
4949

50+
## Prompt Caching
51+
52+
Anthropic and OpenAI support prompt caching, where repeated system prompts are stored and reused at a significant discount - typically 90% cheaper for cached tokens. FastroAI automatically tracks cache tokens and factors them into cost calculations.
53+
54+
When you use FastroAgent, cache tokens are tracked automatically:
55+
56+
```python
57+
response = await agent.run("What's the weather?")
58+
59+
print(f"Input tokens: {response.input_tokens}")
60+
print(f"Cached tokens: {response.cache_read_tokens}") # Tokens from cache (90% discount)
61+
print(f"Cost: ${response.cost_dollars:.6f}") # Accurate cost with cache discount
62+
```
63+
64+
If you're using a long system prompt that gets cached, subsequent requests to the same agent will show cache hits. The cost calculation automatically applies the discounted rate for cached tokens.
65+
66+
**Requirements for prompt caching:**
67+
68+
- **Anthropic:** Prompts >= 1024 tokens with cache_control markers
69+
- **OpenAI:** Prompts >= 1024 tokens on gpt-4o models
70+
5071
## Direct Calculator Usage
5172

5273
Sometimes you need cost calculation without running an agent - estimating costs upfront, building dashboards, or custom tracking:
@@ -66,6 +87,18 @@ print(f"Cost: {cost} microcents") # 7500 microcents
6687
print(f"Cost: ${calc.microcents_to_dollars(cost):.6f}") # $0.007500
6788
```
6889

90+
For accurate costs with prompt caching, pass cache tokens:
91+
92+
```python
93+
cost = calc.calculate_cost(
94+
model="claude-3-5-sonnet",
95+
input_tokens=1000,
96+
output_tokens=500,
97+
cache_read_tokens=800, # 800 of 1000 input tokens were cached
98+
)
99+
# Cost is lower because 800 tokens are priced at 90% discount
100+
```
101+
69102
Model names get normalized automatically. Both `"gpt-4o"` and `"openai:gpt-4o"` work.
70103

71104
### Conversion Methods
@@ -111,6 +144,20 @@ calc.add_pricing_override(
111144
)
112145
```
113146

147+
You can also specify cache token rates for models that support prompt caching:
148+
149+
```python
150+
calc.add_pricing_override(
151+
model="my-cached-model",
152+
input_per_mtok=3.00,
153+
output_per_mtok=15.00,
154+
cache_read_per_mtok=0.30, # 90% discount for cached tokens
155+
cache_write_per_mtok=3.75, # 25% premium for cache writes
156+
)
157+
```
158+
159+
If you don't specify cache rates, default discounts apply (90% for reads, 25% premium for writes) when cache tokens are passed to `calculate_cost()`.
160+
114161
Overrides take precedence over genai-prices. Prices are in dollars per million tokens (the standard unit providers use).
115162

116163
### Using Custom Pricing with Agents

docs/guides/fastro-agent.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,14 +102,21 @@ print(f"Time: {response.processing_time_ms}ms")
102102
| `model` | `str` | Model that generated the response |
103103
| `input_tokens` | `int` | Prompt tokens consumed |
104104
| `output_tokens` | `int` | Completion tokens generated |
105+
| `cache_read_tokens` | `int` | Tokens read from prompt cache (typically 90% cheaper) |
106+
| `cache_write_tokens` | `int` | Tokens written to prompt cache |
105107
| `cost_microcents` | `int` | Cost in 1/1,000,000 of a dollar (for calculations) |
106108
| `cost_dollars` | `float` | Cost in dollars (for display) |
107109
| `processing_time_ms` | `int` | Wall-clock time |
108110
| `tool_calls` | `list` | Tools invoked during generation |
111+
| `tool_call_count` | `int` | Number of tool invocations |
112+
| `request_count` | `int` | Number of API requests (increases with tool use) |
109113
| `trace_id` | `str` | Tracing correlation ID |
114+
| `usage_details` | `dict` | Provider-specific details (e.g., reasoning tokens) |
110115

111116
Use `cost_microcents` when you need to aggregate costs across many calls - integer math won't drift. Use `cost_dollars` when displaying to users.
112117

118+
When prompt caching is enabled (Anthropic, OpenAI with long prompts), `cache_read_tokens` shows how many tokens came from cache at a 90% discount. This is automatically factored into `cost_microcents`.
119+
113120
## Structured Output
114121

115122
Getting strings back means you have to parse them. For anything structured - extracting entities, classifying content, generating schemas - use Pydantic models instead:

0 commit comments

Comments
 (0)