increasing the GPU usage

[acn-reginald-casela]

Is there a way to fully utilize the GPU memory that I'm using? I've deployed an app in GKE that uses a NVIDIA Tesla T4 and it's only using roughly 3-4gb out of 16gb. I'm using the large-v2 model and the usual runtime for a 10-15s clip is around 2-5s.

I was wondering if using the num_workers parameter could also help with GPU utilization and decrease the runtime further to at least 0.5-1sec.

[trungkienbkhn]

No, you can increase num_workers with GPU utilization but it will not increase the throughput (#100 (comment))

[tictproject]

@acn-reginald-casela were you able to solve this? I'm also receiving bad results on Tesla T4 2CPU 15GB each.