Feature Request: Budget limits to cap token usage and costs #1533
Description
Problem
Users running scans against commercial LLM APIs cannot set spending limits or token caps. This creates risk of:
- Unexpected high costs from long-running scans
- No way to enforce budget constraints
- Runaway costs, especially for users new to the tool
- Inability to safely experiment with different configurations
User Feedback
"I'm trying to reduce the cost of running GARAK scans against commercial LLM APIs, where clients are charged based on token usage. The goal is to keep costs low while still getting good security coverage..."
— 33ma (Discord, April 2025)
Impact
- Commercial API costs can reach hundreds of dollars per scan
- No guardrails for cost control
- Users hesitant to run comprehensive scans due to cost uncertainty
- Enterprise adoption blocked by lack of budget controls