Feature Request: Track and display token usage during scans #1532
Description
Problem
Users running garak scans against commercial LLM APIs have no visibility into token consumption. This makes it difficult to:
- Understand how many tokens (prompt vs completion) are being used
- Optimize scan configurations for efficiency
- Compare token efficiency across different probes or settings
- Make informed decisions about scan parameters
User Feedback
"Does GARAK have any way to track or estimate how many tokens are used in each scan?"
— 33ma (Discord, April 2025)
"hi! Is there any way to view total token usage of a scan?"
— mike157 (Discord, September 2025)
Maintainer response: "There is currently not a built in supported path for this... output token counts however are entirely target specific and could vary wildly."
Impact
- Users cannot measure scan efficiency
- No data to inform optimization decisions
- Blind to actual resource consumption
- Difficult to benchmark different scan approaches