Commit 7806273
Fix(llm): complete response cache — set_model invalidation, stats logging, sync mutex (#290)
* fix(llm): complete response cache — set_model invalidation, stats logging, sync mutex
# Conflicts:
# src/llm/response_cache.rs
* fix(llm): address response cache review comments
- Add total_hit_count AtomicU64 that is never decremented on eviction;
maybe_log_stats now uses this counter so hit_rate_pct stays accurate
under high eviction pressure
- Log cache stats before returning on provider error so milestone
intervals (every 100 requests) are never silently skipped
- Add tracing-test dev-dep and three new tests: total_hits_survives_eviction,
stats_logged_at_request_100, stats_logged_on_provider_error_at_interval
- Update PR description to reflect actual set_model() behavior (key
isolation, not cache clear)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 26d274a commit 7806273
3 files changed
Lines changed: 356 additions & 33 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
| 177 | + | |
177 | 178 | | |
178 | 179 | | |
179 | 180 | | |
| |||
0 commit comments