Feature Request: Estimate and report scan costs #1534
Description
Problem
Users have no way to estimate or track the monetary cost of running scans. This makes it difficult to:
- Plan budgets for security testing
- Compare cost-effectiveness of different probes
- Justify scan expenses to stakeholders
- Avoid bill shock from commercial API usage
User Feedback
"I'm trying to reduce the cost of running GARAK scans against commercial LLM APIs, where clients are charged based on token usage."
— 33ma (Discord, April 2025)
"Are there specific test modules that give good results without using a lot of tokens?"
— 33ma (Discord, April 2025)
Impact
- Users cannot predict costs before running scans
- No visibility into which probes are cost-effective
- Hard to budget for regular security testing
- Commercial API bills can surprise users (hundreds of dollars)