-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Description
When users deploy on owned or local GPUs (rather than cloud-rented), power consumption becomes a significant operational cost and constraint that isn't captured in current pricing models.
Requirements
-
Power budget constraint - Allow users to specify a power budget (in watts or kW) that the deployment must stay within. This could be:
- A hard limit (reject configurations exceeding budget)
- A soft preference (penalize configurations in scoring)
-
Power-aware cost calculation - Include electricity costs in the total cost of ownership:
- GPU TDP (Thermal Design Power) data for each GPU type
- User-provided electricity rate ($/kWh)
- Utilization-adjusted power draw estimates
- Monthly power cost = GPU count × TDP × utilization × hours × rate
Additional considerations
- Cooling overhead - Data center cooling typically adds 30-50% to raw power consumption (PUE factor)
- Multi-GPU power scaling - Power draw doesn't scale linearly with GPU count due to interconnect overhead
- Idle vs active power - GPUs draw less power when idle; consider expected utilization patterns
- Power efficiency scoring - Add power efficiency as a potential ranking dimension (tokens/watt)
Data needed
- GPU TDP values for each supported GPU type (already partially available in hardware profiles)
- Typical utilization-adjusted power consumption benchmarks
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels