Skip to content

Sweep mode with --max-seconds fails ungracefully if calculated RPS is too low #115

Open
@sjmonson

Description

@sjmonson

If sweep mode calculates an RPS that results in less then one request being sent out in --max-second range then there is a failure during processing.

Traceback (most recent call last):
  File "/opt/app-root/bin/guidellm", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/guidellm/src/guidellm/__main__.py", line 242, in benchmark
    asyncio.run(
  File "/usr/lib64/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/src/guidellm/src/guidellm/benchmark/entrypoints.py", line 105, in benchmark_generative_text
    async for result in benchmarker.run(
  File "/src/guidellm/src/guidellm/benchmark/benchmarker.py", line 243, in run
    benchmark: BenchmarkT = aggregator.compile()
                            ^^^^^^^^^^^^^^^^^^^^
  File "/src/guidellm/src/guidellm/benchmark/aggregator.py", line 606, in compile
    return GenerativeBenchmark.from_stats(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/guidellm/src/guidellm/benchmark/benchmark.py", line 694, in from_stats
    start_time = min(req.start_time for req in total)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: min() iterable argument is empty

Metadata

Metadata

Assignees

Type

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions