Skip to content

Add development maxExportBatchSize configuration to Periodic MetricReader #4895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dashpole wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add development maxExportBatchSize configuration to Periodic MetricReader #4895

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6520ebe
Add development maxExportBatchSize parameter to Periodic exporting Me…
dashpole Feb 18, 2026
3e479c2
update wording of exportIntervalMillis
dashpole Feb 24, 2026
977309e
require full batches
dashpole Feb 25, 2026
a0403e9
clarify that forceflush should export multiple batches
dashpole Feb 25, 2026
f09d9a8
spelling
dashpole Feb 25, 2026
382f8e6
allow skipping export calls if the timeout is already reached
dashpole Feb 26, 2026
97bdf78
Merge branch 'main' into sdk_batching
dashpole Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ release.

### Metrics

- Add additional in-development requirements to metric start timestamps.
([#4807](https://github.com/open-telemetry/opentelemetry-specification/pull/4807))
- Add development `maxExportBatchSize` parameter to Periodic exporting MetricReader.
([#4895](https://github.com/open-telemetry/opentelemetry-specification/pull/4895))

### Logs

### Baggage
Expand Down
1 change: 1 addition & 0 deletions spec-compliance-matrix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ formats is required. Implementing more than one format is optional.
| Metric SDK supports configuring cardinality limit at MeterReader level | | - | + | + | - | | - | | - | - | - | |
| Metric SDK supports configuring cardinality limit per metric (using Views) | | - | + | + | - | | - | | - | - | + | |
| Metric SDK supports per-timeseries cumulative start timestamps | | | | | | | | | | | | |
| The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter | | - | - | - | - | - | - | - | - | - | - | - |

## Logs

Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/cpp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '-'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/dotnet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '+'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/erlang.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '-'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/go.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '-'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/java.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '+'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/js.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '+'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/php.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '?'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/python.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '-'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/ruby.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '?'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/rust.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '-'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
2 changes: 2 additions & 0 deletions spec-compliance-matrix/swift.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,8 @@ sections:
status: '?'
- name: Metric SDK supports per-timeseries cumulative start timestamps
status: '?'
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
status: '-'
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
1 change: 1 addition & 0 deletions spec-compliance-matrix/template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ sections:
- name: Metric SDK supports configuring cardinality limit at MeterReader level
- name: Metric SDK supports configuring cardinality limit per metric (using Views)
- name: Metric SDK supports per-timeseries cumulative start timestamps
- name: The metric SDK's periodic Reader implementation supports the `maxExportBatchSize` parameter
- name: Logs
features:
- name: LoggerProvider.Get Logger
Expand Down
26 changes: 22 additions & 4 deletions specification/metrics/sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1448,9 +1448,23 @@ a user-configurable time interval, and passes the metrics to the configured
Configurable parameters:

* `exportIntervalMillis` - the time interval in milliseconds between two
consecutive exports. The default value is 60000 (milliseconds).
consecutive collections. The default value is 60000 (milliseconds).
* `exportTimeoutMillis` - how long the export can run before it is cancelled.
The default value is 30000 (milliseconds).
* **Status**: [Development](../document-status.md) - `maxExportBatchSize` - the
maximum number of metric data points in a batch that are provided to a single
export.

**Status**: [Development](../document-status.md) - When `maxExportBatchSize` is
configured, the reader MUST ensure no batch provided to `Export` exceeds the
`maxExportBatchSize` by splitting the batch of metric data points into smaller
batches. The initial batch of metric data MUST be split into as many "full"
batches of size `maxExportBatchSize` as possible -- even if this splits up data
points that belong to the same metric into different batches. The reader MUST
ensure all metric data points from a single `Collect()` are provided to
Copy link
Member

@cijothomas cijothomas Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this assuming we won't timeout before finishing all metric datapoints from collect1()?
If we had 1000 points, and 100 is max_batch_size, and we timeout at 8th batch. during next collect2(), we start over, and not continue from 8th batch from previous collect?

Copy link
Contributor Author

@dashpole dashpole Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent is that the timeout is still applied to a single export call, so we would attempt each Export. The problem that could arise is that the SDK generates so many batches that it ends up delaying the next export call. Ideally, language implementations would delay the next Collect, rather than continuously spawning new collections when the previous one hasn't completed.

We could alternatively change the timeout to apply to a collect and export(s) cycle, and adopt behavior similar to what you are suggesting (where once we timeout, we discard all unsent batches).

`Export` before metric data points from a subsequent `Collect()` so that metric
points are sent in-order. The reader MUST NOT combine metrics from different
`Collect()` calls into the same batch provided to `Export`.

The reader MUST synchronize calls to `MetricExporter`'s `Export`
to make sure that they are not invoked concurrently.
Expand All @@ -1475,9 +1489,13 @@ from `MetricReader` and start a background task which calls the inherited
This method provides a way for the periodic exporting MetricReader
so it can do as much as it could to collect and send the metrics.

`ForceFlush` SHOULD collect metrics, call [`Export(batch)`](#exportbatch)
and [`ForceFlush()`](#forceflush-2) on the configured
[Push Metric Exporter](#push-metric-exporter).
`ForceFlush` SHOULD collect metrics, split into batches if necessary, call
[`Export(batch)`](#exportbatch) on each batch and
[`ForceFlush()`](#forceflush-2) on the configured
[Push Metric Exporter](#push-metric-exporter). `ForceFlush` MAY skip
[`Export(batch)`](#exportbatch) calls if the timeout is already expired, but
SHOULD still call [`ForceFlush()`](#forceflush-2) on the configured
[Push Metric Exporter](#push-metric-exporter) even if the timeout has passed.

`ForceFlush` SHOULD provide a way to let the caller know whether it succeeded,
failed or timed out. `ForceFlush` SHOULD return some **ERROR** status if there
Expand Down
Loading