Skip to content

Treat bench() errors as first-class iteration results #71

Description

@wfxr

Problem

If a benchmark iteration returns Err(...), collectors currently only record it in error_dist and do not increment iteration counters or record latency. This can lead to optimistic throughput/success ratios, especially since examples commonly use ? inside bench().

Current behavior

Proposed solution

Option A (compatible, recommended): move duration measurement into the runner so it can always attach a duration to both Ok and Err outcomes, and change the result channel to carry a unified iteration outcome:

pub enum IterOutcome {
    Ok(IterReport),
    Err { duration: Duration, message: String },
}

Collectors would then:

  • Increment iteration counters for Err
  • Record duration into the histogram
  • Add a synthetic Status::error(...) entry into status_dist
  • Keep error_dist for messages

Option B (docs-only): document the semantics clearly and update examples to convert errors to IterReport.status, but keep current behavior.

Acceptance criteria

  • Iterations that fail with Err are reflected in total iteration counts
  • Success ratio and throughput include failures in the denominator
  • Error details remain available in error_dist
  • Examples demonstrate the recommended error-handling pattern

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium - Should be addressed when convenientXLVery large change (multi-week); likely needs an RFCapiPublic API improvementenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions