Skip to content

Produced signals rate in benchmarks is off by as much as 20% #2713

@JakeDern

Description

@JakeDern

Pre-filing checklist

  • I searched existing issues and didn't find a duplicate

Component(s)

Rust OTAP dataflow (rust/otap-dataflow/)

Bug Description

Metrics for all of our benchmarks show that the produced signals rate is off by as much as 20%. For example, in the 100klrps test, the metrics are showing ~120klrps. It also varies from run to run and scenario to scenario which compromises the results.

Image

Steps to Reproduce

View the 100klrps benchmarks at https://open-telemetry.github.io/otel-arrow/benchmarks/continuous/.

Expected Behavior

Almost exactly 100klrps.

Actual Behavior

Closer to 120klrps.

OTel-Arrow Version

NA

Environment

No response

Configuration

Log Output

Additional Context

No response

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions