Include energy and hardware requirements to benchmarked metrics.

[eliemichel]

Hello The Yard! Thank you for this initiative and for sharing such a benchmarking infrastructure.

I'd like to take this opportunity to extend the notion of performance (as in "the thing we want to optimize in our pipeline") beyond the speed/memory/UX/quality: assessing the requirement of an engine in terms of energy expense and choice of hardware is getting increasingly important (you might be aware of eco-conditionality of CNC funding), yet there is (not?) suprisingly very few resource available on the topic when it comes to VFX digital infrastructure.

Your benchmark has the opportunity to become a pioneer in providing such information about environmental impacts, which will with no doubt become valuable in the near future. Happy to help about this, would you want to expand in that direction!

[nicolaspopravka]

Hey, that's a cool idea! How would you go about measuring the energy consumption of the system running the benchmark?

[eliemichel]

If running on Linux, I'd recommend PowerAPI. If not, it is a bit of a pain (I am still actively working on figuring out a good measurement pipeline TBH), you can hack something up using vendor specific tools like Intel PCM, the nvidia-smi command (or underlying NVML library), AMD uProf, etc.

Another approach is to measure the power consumption directly at the power socket level, this makes sure we encompass all sources of energy consumption (motherboard, memory, processor, drives, fans, dissipation, etc.) but it requires a dedicated device, and it is imperfect in case of quickly varrying power.

[nicolaspopravka]

The USD Render Benchmark is, for now, expected to run on Linux only, so PowerAPI sounds like a great fit. I’d definitely welcome any contributions if you're familiar with PowerAPI and would like to extend the benchmark in this direction. Feel free to submit a pull request whenever you’re ready!