-
-
Notifications
You must be signed in to change notification settings - Fork 13
Treat bench() errors as first-class iteration results #71
Copy link
Copy link
Open
Labels
P2Medium - Should be addressed when convenientMedium - Should be addressed when convenientXLVery large change (multi-week); likely needs an RFCVery large change (multi-week); likely needs an RFCapiPublic API improvementPublic API improvementenhancementNew feature or requestNew feature or request
Description
Metadata
Metadata
Assignees
Labels
P2Medium - Should be addressed when convenientMedium - Should be addressed when convenientXLVery large change (multi-week); likely needs an RFCVery large change (multi-week); likely needs an RFCapiPublic API improvementPublic API improvementenhancementNew feature or requestNew feature or request
Problem
If a benchmark iteration returns
Err(...), collectors currently only record it inerror_distand do not increment iteration counters or record latency. This can lead to optimistic throughput/success ratios, especially since examples commonly use?insidebench().Current behavior
IterStats(src/report.rs:41-54)iters/items/bytes(src/collector/silent.rs:59-73,src/collector/tui.rs:200-225)Proposed solution
Option A (compatible, recommended): move duration measurement into the runner so it can always attach a duration to both
OkandErroutcomes, and change the result channel to carry a unified iteration outcome:Collectors would then:
Errdurationinto the histogramStatus::error(...)entry intostatus_disterror_distfor messagesOption B (docs-only): document the semantics clearly and update examples to convert errors to
IterReport.status, but keep current behavior.Acceptance criteria
Errare reflected in total iteration countserror_dist