Generate OTel metrics with context limits by blt · Pull Request #1331 · DataDog/lading

blt · 2025-04-24T21:36:15Z

What does this PR do?

Generate OTel metrics with context limits

This commit allows users to configure an otel metrics payload that
has a fixed number of 'contexts' to reuse the term from dogstatsd.
This is done by setting a lot of fiddly knobs and the user is
responsible for keeping track of how many unique instances are
possible.

This commit will be chased with some clean-up as I left a lot of
internal duplication in the code. I am also unsure if this generates
sensible load run against a collector, just that it obeys the
protocol by construction.

REF SMPTNG-659

blt · 2025-04-24T21:36:31Z

Introduce tag generator into OTel metrics payload #1340 : 2 dependent PRs (#1341 , #1342 )
Add configuration validation for OTel metrics #1339
Bound contexts in Opentelemetry Metrics #1337
Generate units according to OTel required spec #1335
Generate OTel metrics with context limits #1331 👈 (View in Graphite)
Convert opentelemetry_metric to use MemberGenerator method #1326
Make tagset generation generic #1336
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

This commit allows users to configure an otel metrics payload that has a fixed number of 'contexts' to reuse the term from dogstatsd. This is done by setting a lot of fiddly knobs and the user is responsible for keeping track of how many unique instances are possible. This commit will be chased with some clean-up as I left a lot of internal duplication in the code. I am also unsure if this generates sensible load run against a collector, just that it obeys the protocol by construction. REF SMPTNG-659 Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

This commit avoids non-trivial duplication of the same string pool allocation. Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

dd-devflow · 2025-05-14T22:44:05Z

View all feedbacks in Devflow UI.

2025-05-14 22:44:05 UTC ℹ️ Start processing command /merge

2025-05-14 22:44:09 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2025-05-15 02:45:04 UTC ⚠️ MergeQueue: This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

scottopell · 2025-05-15T15:42:10Z

lading_payload/src/opentelemetry_metric.rs

+            None
+        };
+
+        let total_scopes = self.scopes_per_resource.sample(rng);


if resource is None, then we still generate scope metrics. Is this intentional? The naming of scopes_per_resource makes me question if this is correct (ie, if we have a no resource, then there should be no scopes)

scottopell · 2025-05-15T15:43:49Z

lading_payload/src/opentelemetry_metric.rs

+        Ok(v1::ScopeMetrics {
+            scope: Some(instrumentation_scope),
+            metrics,
+            schema_url: String::new(), // 0-sized alloc


what is a zero-sized alloc? Asking more from a curiosity of how rust handles this

scottopell

looks good overall, a few thoughts in-line, but approving as they may be addressed upstack

blt · 2025-05-15T15:45:35Z

Merge activity

May 15, 11:45 AM EDT: @blt merged this pull request with Graphite.

blt mentioned this pull request Apr 24, 2025

Migrate OTel/dogstatsd common functions to src/common #1329

Merged

blt mentioned this pull request Apr 24, 2025

Convert opentelemetry_metric to use MemberGenerator method #1326

Merged

blt added the no-changelog label Apr 24, 2025 — with Graphite App

blt marked this pull request as ready for review April 24, 2025 21:36

blt requested a review from a team as a code owner April 24, 2025 21:36

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 089e4d7 to 74cc380 Compare April 28, 2025 19:05

blt mentioned this pull request Apr 28, 2025

Generate units according to OTel required spec #1335

Merged

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 74cc380 to eae62f5 Compare April 28, 2025 19:24

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from b17ae82 to 5d2d1c0 Compare April 28, 2025 19:24

blt mentioned this pull request Apr 29, 2025

Make tagset generation generic #1336

Merged

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from 5d2d1c0 to b01874b Compare April 29, 2025 17:02

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 19d131a to 6a90429 Compare April 29, 2025 17:02

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 6a90429 to f18df04 Compare May 8, 2025 18:46

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from b01874b to aaef818 Compare May 8, 2025 18:46

This was referenced May 8, 2025

Introduce blockgen as an optimization target #1346

Closed

Introduce RejectSet #1347

Closed

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from aaef818 to 9ffbdec Compare May 8, 2025 19:02

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from f18df04 to 6601f83 Compare May 8, 2025 19:02

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from 9ffbdec to 9d8800f Compare May 8, 2025 19:14

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 6601f83 to 6d042b6 Compare May 8, 2025 19:14

blt force-pushed the blt/otel_metrics_common_dogstatsd branch from 9d8800f to bb9ab4d Compare May 12, 2025 18:02

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 6d042b6 to 10f6fca Compare May 12, 2025 18:02

This was referenced May 12, 2025

Update payloadtool, block cache creation to have additional telemetry #1349

Merged

OTel Metric templates are direct protobuf types #1350

Merged

blt force-pushed the blt/convert_opentelemetry_metric_to_use_membergenerator_method branch 2 times, most recently from 7a78bb2 to 7bfe432 Compare May 14, 2025 16:52

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch 2 times, most recently from 2687544 to da8aa60 Compare May 14, 2025 17:02

blt force-pushed the blt/convert_opentelemetry_metric_to_use_membergenerator_method branch from 7bfe432 to d6f15cb Compare May 14, 2025 21:04

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from da8aa60 to 9293145 Compare May 14, 2025 21:04

blt changed the base branch from blt/convert_opentelemetry_metric_to_use_membergenerator_method to graphite-base/1331 May 14, 2025 21:48

blt added 4 commits May 14, 2025 21:58

extract generate_attributes

2f7c220

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

Rc the str_pool

d4dc4d1

This commit avoids non-trivial duplication of the same string pool allocation. Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

fix rebase goofs

f3d13aa

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from 9293145 to ee13a5a Compare May 14, 2025 21:58

blt force-pushed the graphite-base/1331 branch from d6f15cb to d4f3c65 Compare May 14, 2025 21:58

graphite-app bot changed the base branch from graphite-base/1331 to main May 14, 2025 21:59

test disable

db2e90d

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>

blt force-pushed the blt/generate_otel_metrics_with_context_limits branch from ee13a5a to db2e90d Compare May 14, 2025 21:59

dd-devflow bot added mergequeue-status: waiting mergequeue-status: removed and removed mergequeue-status: waiting labels May 14, 2025

scottopell reviewed May 15, 2025

View reviewed changes

scottopell approved these changes May 15, 2025

View reviewed changes

blt merged commit 0c990f7 into main May 15, 2025
24 of 25 checks passed

blt deleted the blt/generate_otel_metrics_with_context_limits branch May 15, 2025 15:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate OTel metrics with context limits#1331

Generate OTel metrics with context limits#1331
blt merged 5 commits intomainfrom
blt/generate_otel_metrics_with_context_limits

blt commented Apr 24, 2025 •

edited

Loading

Uh oh!

blt commented Apr 24, 2025 •

edited

Loading

Uh oh!

dd-devflow bot commented May 14, 2025 •

edited

Loading

Uh oh!

scottopell May 15, 2025

Uh oh!

scottopell May 15, 2025

Uh oh!

scottopell left a comment

Uh oh!

Uh oh!

blt commented May 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

blt commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

blt commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-devflow bot commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

scottopell May 15, 2025

Choose a reason for hiding this comment

Uh oh!

scottopell May 15, 2025

Choose a reason for hiding this comment

Uh oh!

scottopell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

blt commented May 15, 2025

Merge activity

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

blt commented Apr 24, 2025 •

edited

Loading

blt commented Apr 24, 2025 •

edited

Loading

dd-devflow bot commented May 14, 2025 •

edited

Loading