Skip to content

Optimization passes for Otel metrics payload generation#1342

Merged
blt merged 6 commits intomainfrom
blt/optimization_passes_for_otel_metrics_payload_generation
May 15, 2025
Merged

Optimization passes for Otel metrics payload generation#1342
blt merged 6 commits intomainfrom
blt/optimization_passes_for_otel_metrics_payload_generation

Conversation

@blt
Copy link
Copy Markdown
Collaborator

@blt blt commented May 1, 2025

What does this PR do?

This commit bundles a small number of optimization passes on the payload generation code. We're clocking in at 100Mb/s after this work -- measured by opentelemetry_metric_all/1000000 -- up from 54Mb/s so it's something. Still noticably slow at high entropy.

Motivation

SMPTNG-659

@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 1fdd500 to d0bd6ac Compare May 1, 2025 22:39
@blt blt added the no-changelog label May 1, 2025 — with Graphite App
@blt blt marked this pull request as ready for review May 1, 2025 22:39
@blt blt requested a review from a team as a code owner May 1, 2025 22:39
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from 7fb6ccc to 290b8f4 Compare May 8, 2025 18:46
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from a08d215 to 19ce922 Compare May 8, 2025 18:46
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 19ce922 to cc7a0fc Compare May 8, 2025 18:49
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from 290b8f4 to f0996d7 Compare May 8, 2025 19:02
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch 2 times, most recently from 462e549 to 6820721 Compare May 8, 2025 19:14
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from f0996d7 to 65e9d76 Compare May 8, 2025 19:14
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 6820721 to 9036751 Compare May 8, 2025 23:00
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from 65e9d76 to f9c8702 Compare May 12, 2025 18:02
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 2eae615 to 935e384 Compare May 12, 2025 18:02
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from f9c8702 to 0e5e725 Compare May 12, 2025 21:43
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from 38c88c5 to 316ebaf Compare May 15, 2025 15:47
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 0570d75 to 986d6dc Compare May 15, 2025 15:48
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from 316ebaf to c2ded89 Compare May 15, 2025 15:59
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 986d6dc to 810c9be Compare May 15, 2025 15:59
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from c2ded89 to c2a5afe Compare May 15, 2025 17:25
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 810c9be to 533936d Compare May 15, 2025 17:25
codegen-units = 1 # Increases compile time but improves optimization alternatives.
opt-level = 3 # Optimize with 'all' optimization flipped on. May produce larger binaries than 's' or 'z'.
panic = "abort"
debug = false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really want to disable debug? Doesn't this come into play for symbolizing the stack on panic?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah this is me actually just expressing what the default is. I'll fix this up-stack. You make a good point.

@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 533936d to cc26354 Compare May 15, 2025 22:59
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch from c2a5afe to 44ee1a3 Compare May 15, 2025 22:59
@blt blt force-pushed the blt/introduce_tag_generator_into_otel_metrics_payload branch 2 times, most recently from e610b10 to 11841e5 Compare May 15, 2025 23:15
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from cc26354 to dbb3152 Compare May 15, 2025 23:15
@blt blt changed the base branch from blt/introduce_tag_generator_into_otel_metrics_payload to graphite-base/1342 May 15, 2025 23:29
blt added 5 commits May 15, 2025 23:29
This commit bundles a small number of optimization passes on the payload
generation code. We're clocking in at 100Mb/s after this work -- measured by
opentelemetry_metric_all/1000000 -- up from 54Mb/s so it's something. Still
noticably slow at high entropy.

SMPTNG-659

Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from dbb3152 to 40ad9a2 Compare May 15, 2025 23:29
@blt blt force-pushed the graphite-base/1342 branch from 11841e5 to 231f9b9 Compare May 15, 2025 23:29
@preinlein preinlein changed the base branch from graphite-base/1342 to main May 15, 2025 23:29
Signed-off-by: Brian L. Troutwine <brian.troutwine@datadoghq.com>
@blt blt force-pushed the blt/optimization_passes_for_otel_metrics_payload_generation branch from 40ad9a2 to 68c97ab Compare May 15, 2025 23:29
@blt blt merged commit ed5fbe0 into main May 15, 2025
21 checks passed
Copy link
Copy Markdown
Collaborator Author

blt commented May 15, 2025

Merge activity

  • May 15, 7:44 PM EDT: @blt merged this pull request with Graphite.

@blt blt deleted the blt/optimization_passes_for_otel_metrics_payload_generation branch May 15, 2025 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants