Skip to content

Conversation

@sephib
Copy link

@sephib sephib commented Nov 27, 2025

Summary

Fixes a bug when an HTML benchmark reports is generated. The HTML report crashes in the browsers with e.g. Error: xs[2] == xs[3] when rendering distributions with consecutive duplicate percentile values.

This occurs when benchmarks have limited data points (e.g., < 10 requests), causing multiple percentiles to collapse to the same value. The visualization library expects strictly increasing x-values and fails when encountering duplicates.

Root Cause

When a benchmark has very few successful requests (e.g., 3 requests), percentile calculations result in many duplicate values:

# Before fix - causes visualization error
{
  "p001": 15.288,
  "p01": 15.288,   # duplicate
  "p05": 15.288,   # duplicate
  "p10": 15.288,   # duplicate
  "p25": 15.288,   # duplicate
  "p50": 16.413,   # unique
  "p75": 16.413,   # duplicate
  "p90": 17.035,   # unique
  ...
}

# After fix - renders successfully
{
  "p001": 15.288,
  "p50": 16.413,
  "p90": 17.035
}

Details

  • Added _filter_duplicate_percentiles() helper function to remove consecutive duplicate percentile values
  • Override model_dump() in _TabularDistributionSummary to automatically apply filtering during HTML generation
  • Added comprehensive unit tests (8 test cases) covering all edge cases

Test Plan

Unit Tests

uv run pytest tests/unit/benchmark/test_html_output.py -v
# 8/8 tests passing

  • [ X] "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • [ x] Includes AI-assisted code completion
  • [ x] Includes code generated by an AI application
  • [ x] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@sephib sephib marked this pull request as ready for review November 27, 2025 20:59
@sjmonson sjmonson added this to the v0.4.1 milestone Dec 2, 2025
@sjmonson
Copy link
Collaborator

sjmonson commented Dec 2, 2025

See failed workflows, can be run locally with pre-commit.

@sephib
Copy link
Author

sephib commented Dec 7, 2025

Hi @sjmonson

  1. i've rebased the branch
  2. run pre-commit - passes
  3. there are some files that do not pass (but not related to this PR)
        modified:   src/guidellm/data/builders.py
        modified:   src/guidellm/data/config.py
        modified:   src/guidellm/data/entrypoints.py
        modified:   src/guidellm/data/schemas.py
        modified:   tests/unit/data/deserializers/test_synthetic.py
        modified:   tests/unit/data/test_builders.py

Please let me know if there any other issues to attend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants