Skip to content

Commit e8d9da1

Browse files
authored
Update vllm_client.py (#543)
Extend vllm request time out so runs stop failing for no good reason. <!-- markdownlint-disable --> ## Purpose Increase timeout on the vllm client for slower training runs. ## Description ## Related Issue ## Tests I have filled in: - [x] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)". - [ ] The test plan/results, such as providing test command and pasting the results. - [ ] (Optional) The necessary documentation update. - [x] I (a human) have written or reviewed the code in this pr to the best of my ability. Signed-off-by: MeganEFlynn <mflynn@redhat.com>
1 parent 8e023e0 commit e8d9da1

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/speculators/data_generation/vllm_client.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
logger = logging.getLogger(__name__)
99

10-
DEFAULT_REQUEST_TIMEOUT = 15 # seconds
10+
DEFAULT_REQUEST_TIMEOUT = 120 # seconds
1111
DEFAULT_MAX_RETRIES = 3
1212
RETRY_BACKOFF_BASE = 2 # seconds
1313

0 commit comments

Comments
 (0)