how to speed up the evaluation

[uobinxiao]

Thank you very much for sharing this useful benchmark. Currently, I am using this benchmark to evaluate our foundation model. However, I’ve noticed that both GPU and CPU utilization remain very low throughout the process. As a result, the overall evaluation takes very long time to complete. Is there anyway to speed up the evaluation?

[nkaenzig]

Hi @uobinxiao, thanks for raising this question. Can you please provide more information on the type of benchmarks where you observe the low GPU utilisation . Also, does this happen during the predict or fit step, or both?
E.g. a common issue for low GPU usage during predict is when loading the data from a slow disk, i.e. the IO bandwidth of the disk in that case becomes a bottleneck.