[Runner] Never pad a step past max_num_batched_tokens - #3466

[Runner] Never pad a step past max_num_batched_tokens#3466

gutianyu-google wants to merge 2 commits into

vllm-project:mainvllm-project/tpu-inference:mainfrom

gutianyu-google:fix/token-paddings-include-max-batchedgutianyu-google/tpu-inference:fix/token-paddings-include-max-batchedCopy head branch name to clipboard