Skip to content

[Feature] Add a customizable timeout for GuideLLM #84

@thameem-abbas

Description

@thameem-abbas

For ultra-long input sequence lengths, GuideLLM can take long times to prepare the samples for the benchmark. The default multiplier of 1.5x of test duration is insufficient in this case and we will need to provide the ability to tweak the timeout duration here.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions