Skip to content

Add power consumption tracking and budget constraints for owned/local GPUs #70

@anfredette

Description

@anfredette

When users deploy on owned or local GPUs (rather than cloud-rented), power consumption becomes a significant operational cost and constraint that isn't captured in current pricing models.

Requirements

  1. Power budget constraint - Allow users to specify a power budget (in watts or kW) that the deployment must stay within. This could be:

    • A hard limit (reject configurations exceeding budget)
    • A soft preference (penalize configurations in scoring)
  2. Power-aware cost calculation - Include electricity costs in the total cost of ownership:

    • GPU TDP (Thermal Design Power) data for each GPU type
    • User-provided electricity rate ($/kWh)
    • Utilization-adjusted power draw estimates
    • Monthly power cost = GPU count × TDP × utilization × hours × rate

Additional considerations

  • Cooling overhead - Data center cooling typically adds 30-50% to raw power consumption (PUE factor)
  • Multi-GPU power scaling - Power draw doesn't scale linearly with GPU count due to interconnect overhead
  • Idle vs active power - GPUs draw less power when idle; consider expected utilization patterns
  • Power efficiency scoring - Add power efficiency as a potential ranking dimension (tokens/watt)

Data needed

  • GPU TDP values for each supported GPU type (already partially available in hardware profiles)
  • Typical utilization-adjusted power consumption benchmarks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions