fix: cuda_graph_config optimized with less batch_sizes and enable_padding to True

[jasonqinzhou]

Overview:

Decode warmup step takes too long with too many batch sizes. It will also crash the deployment due to memory limit.
Reduce the batch_sizes to exponential and enable_padding for intermittent batch sizes.

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

[tianhaox]

this will hugely impact decode perf, we need to be smarter with this setting, do you see any problem with current setting? maybe we can prune some batchsize when bs is larger than a specifc value, say, 128

[jasonqinzhou]

this will hugely impact decode perf, we need to be smarter with this setting, do you see any problem with current setting? maybe we can prune some batchsize when bs is larger than a specifc value, say, 128

The problem is this process takes a very long time for large batch size and it could also crash the deployment with OOM.
Sure we can try less aggressive approach.