Skip to content

feat(om2): add native histograms to OpenMetrics2.0 #2634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
krajorama wants to merge 65 commits into
base: main
Choose a base branch
from
Open

feat(om2): add native histograms to OpenMetrics2.0 #2634

krajorama wants to merge 65 commits into from
+238 −29

Conversation

@krajorama
Copy link
Member

@krajorama krajorama commented Apr 24, 2025 •
edited
Loading

Background

Based on https://github.com/prometheus/proposals/blob/main/proposals/2024-01-29_native_histograms_text_format.md
And OpenMetrics 2.0 WG discussions.

Changes

  • Allow structured complex types marked by "{" and "}" in the specification.
  • Allow multiple exemplars per complex type value.
  • Require that exemplars for complex type values have the timestamp. (1)
  • Be permissive about observing NaN , +Inf, -Inf. Discourage observing NaN.
  • Split histogram into ones with classic and native buckets.
  • For classic buckets, define behavior when observing NaN.
  • Define the native buckets and also how NaN is handled.
  • Define the text format of native histograms and also their exemplars.
  • Be permissive about the existence of _sum and _count series for classic histograms - for backwards compatibility.

(1) Not doing this on account of being an implementation limitation and also contradicts having backwards compatibility when exposing classic histograms as complex type.

Out of scope

Open questions / decisions

See OpenMetrics2.0 WG meeting notes tab

@krajorama
feat(om2): add native histograms to OpenMetrics2.0
ddbace4 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
define the counter histogram model
2f4113f 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
small fixes to histogram model
a47ce0c 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
update gauge histogram model
10dc43d 
Signed-off-by: György Krajcsovits <[email protected]>
krajorama
krajorama commented Apr 30, 2025
View reviewed changes
content/docs/specs/om/open_metrics_spec_2_0.md Outdated

Numbers MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero.

Complex data types MUST contain all information necessary to recreate a Metric Type, with the exception of Created time and Exemplars.
Copy link
Member Author

@krajorama krajorama Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assume we'll have the created timestamp separate from the JSON like data. prometheus/OpenMetrics#285

@krajorama
wip: wip
d12dfeb 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Copy link
Member Author

krajorama commented May 6, 2025

Note self: add details how summaries and classic histograms one liners (so not NHCB spans/deltas) fit into it.

@krajorama krajorama mentioned this pull request May 8, 2025
Open
krajorama added 4 commits May 22, 2025 10:05
@krajorama
adjust wording
471a7dd 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
add abnf and presentation
ec991a4 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
updates
dc308e2 
Add semantic conventions about where complex types may occure.
Allow empty spans and deltas.
Be more precise.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Add gauge histogram syntax
57c6ab8 
Signed-off-by: György Krajcsovits <[email protected]>
ArthurSens
ArthurSens reviewed May 24, 2025
View reviewed changes
Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had some minutes to read the PR. Couldn't read everything though, I'll do a more complete review another day!

Reviewed from mobile

@beorn7 beorn7 self-requested a review May 27, 2025 11:26
krajorama added 2 commits May 27, 2025 15:53
@krajorama
clarify NaN and Inf and be permissive
32a65cf 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
fix from Arthur's comments
6968e00 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama krajorama marked this pull request as ready for review May 27, 2025 14:01
@krajorama

This comment was marked as outdated.

Sign in to view
krajorama
krajorama commented Jun 3, 2025
View reviewed changes
@krajorama krajorama mentioned this pull request Jun 4, 2025
Merged
krajorama added 2 commits June 11, 2025 09:03
@krajorama
test(om2): test that examples follow the spec and fix if not
4da9670 
I did not want to clutter the examples with adding "# EOF" to all,
so I made a new marker to explicitly add it on request.

There was only one faulty example where we had an extra space, see
the Exemplars section.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Update with exemplars
7965787 
Allow multiple exemplars for complex types, i.e. native histograms.
But require that the timestamp is present.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama krajorama force-pushed the krajo/om2.0-native-histograms branch from c16426b to 7965787 Compare June 11, 2025 07:27
@krajorama krajorama mentioned this pull request Jun 12, 2025
Open
beorn7
beorn7 requested changes Jun 12, 2025
View reviewed changes
Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing all of this.

Mostly commented about the histogram aspect (but I couldn't resist and referred to OM specific things now and then).

@krajorama
Merge remote-tracking branch 'origin/main' into krajo/om2.0-native-hi…
738d49f 
…stograms

# Conflicts:
#	docs/specs/om/open_metrics_spec_2_0.md
@krajorama
remove stateful statement on classic and native buckets
341f4d2 
Signed-off-by: György Krajcsovits <[email protected]>
@bwplotka bwplotka mentioned this pull request Nov 12, 2025
Closed
@krajorama
Merge remote-tracking branch 'origin/main' into krajo/om2.0-native-hi…
4269914 
…stograms
@krajorama
rephrase wording on NaN observations
fd41d04 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Reword rules on +-Inf, there no reason to refuse measuring such values
f414261 
as we have buckets where we can count such measurements, as opposed to
NaN.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Move rules on exemplar timestamps out of histogram discussion
16af634 
It will handled on float vs complex type sample level.

Signed-off-by: György Krajcsovits <[email protected]>
@krajorama krajorama mentioned this pull request Nov 26, 2025
Open
bwplotka
bwplotka reviewed Dec 3, 2025
View reviewed changes
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!

Looks good, just some suggestions for tighter language 💪🏽

krajorama and others added 3 commits December 3, 2025 11:48
@krajorama @bwplotka
Apply suggestions from code review
973863f 
Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: George Krajcsovits <[email protected]>
@krajorama @bwplotka
Update docs/specs/om/open_metrics_spec_2_0.md
4cab5f2 
Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: George Krajcsovits <[email protected]>
@krajorama
Do not refer to the format from the data model
8071a29 
Signed-off-by: György Krajcsovits <[email protected]>
krajorama
krajorama commented Dec 3, 2025
View reviewed changes
krajorama
krajorama commented Dec 3, 2025
View reviewed changes
krajorama
krajorama commented Dec 3, 2025
View reviewed changes
krajorama
krajorama commented Dec 3, 2025
View reviewed changes
dashpole
dashpole reviewed Dec 3, 2025
View reviewed changes
krajorama and others added 7 commits December 3, 2025 17:20
@krajorama
document why we cannot have "le" label alongside classic buckets
f47b2b5 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama @dashpole
Apply suggestions from code review
ef32f70 
Co-authored-by: David Ashpole <[email protected]>
Signed-off-by: George Krajcsovits <[email protected]>
@krajorama
reword statement about classic bucket threshold being negative
0b46a71 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama @dashpole
Apply suggestions from code review
273b1b2 
Co-authored-by: David Ashpole <[email protected]>
Signed-off-by: George Krajcsovits <[email protected]>
@krajorama
fixes from comments
535e004 
Signed-off-by: György Krajcsovits <[email protected]>
@krajorama
Merge remote-tracking branch 'origin/main' into krajo/om2.0-native-hi…
e0e7d9d 
…stograms
@krajorama
Reword schemas
5eb6197 
Signed-off-by: György Krajcsovits <[email protected]>
@jan--f jan--f added the kind/feature New documentation content label Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@beorn7 beorn7 beorn7 left review comments

@bwplotka bwplotka bwplotka left review comments

@ArthurSens ArthurSens ArthurSens left review comments

@dashpole dashpole dashpole approved these changes

Assignees

No one assigned

Labels

kind/feature New documentation content

Projects

Open Metrics 2.0
Status: In Progress

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@krajorama @beorn7 @bwplotka @dashpole @ArthurSens @jan--f