[Misc] Batch: Extend batch session retry handling with capped backoff to support more retry times. - #2658

[Misc] Batch: Extend batch session retry handling with capped backoff to support more retry times.#2658

zhangjyr wants to merge 2 commits into

vllm-project:mainvllm-project/aibrix:mainfrom

zhangjyr:feature/extended_session_retrialzhangjyr/aibrix:feature/extended_session_retrialCopy head branch name to clipboard