Skip to content

add option to not increment endpoint retry limit - #3079

Open
jveronvialard wants to merge 1 commit into
mainfrom
jveronvialard/endpoint-retries
Open

add option to not increment endpoint retry limit#3079
jveronvialard wants to merge 1 commit into
mainfrom
jveronvialard/endpoint-retries

Conversation

@jveronvialard

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a no_infinite_endpoint_retries option for vLLM endpoints. When enabled, retryable endpoint errors no longer increase max_num_tries, so requests fail after the fixed retry limit instead of retrying indefinitely.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

Signed-off-by: Julien Veron Vialard <jveronvialar@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cmunley1

cmunley1 commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

/ok to test 0859232

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants