Skip to content

Commit 3ce308b

Browse files
authored
[Geneva Exporter] Implement concurrent uploads and add metrics (open-telemetry#1653)
Refactor GenevaExporter to support concurrent batch uploads, to improve throughput for high-volume telemetry - allowing multiple in-flight upload requests per exporter instance (controlled by max_concurrent_uploads). It also introduces internal metrics to track the health and performance of the export process and ensures proper Ack/Nack propagation. Also added tests to ensure that the exporter correctly parses the user's YAML configuration, ensuring all fields and authentication methods (like certificates or managed identities) are deserialized properly.
1 parent 2c5e357 commit 3ce308b

3 files changed

Lines changed: 289 additions & 107 deletions

File tree

rust/otap-dataflow/crates/otap/src/experimental/geneva_exporter/README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,12 @@ You should see `urn:otel:geneva:exporter` in the Exporters list.
3131
./target/release/df_engine --pipeline config.yaml --num-cores 4
3232
```
3333

34+
### Notes on throughput knobs
35+
36+
- `max_concurrent_uploads` limits how many batches the exporter will upload concurrently.
37+
- `max_buffer_size` is currently reserved for a future buffering/flush implementation.
38+
It is accepted by config parsing but does not change runtime behavior yet.
39+
3440
## Test Configuration
3541

3642
To test using the configuration file `otlp-geneva.yaml` provided
@@ -40,7 +46,7 @@ in this directory:
4046
# Start the collector
4147
./target/release/df_engine \
4248
--pipeline crates/otap/src/experimental/geneva_exporter/\
43-
test-config-otlp-receiver.yaml \
49+
otlp-geneva.yaml \
4450
--num-cores 1
4551

4652
# In another terminal, send test data:

0 commit comments

Comments
 (0)