Skip to content

Add support for DogStatsD v1.1 - value packing#762

Merged
tiedotguy merged 3 commits intoatlassian:masterfrom
Wheezyx:mwedel/add-dgostatsdv11-support
Jul 28, 2025
Merged

Add support for DogStatsD v1.1 - value packing#762
tiedotguy merged 3 commits intoatlassian:masterfrom
Wheezyx:mwedel/add-dgostatsdv11-support

Conversation

@Wheezyx
Copy link
Contributor

@Wheezyx Wheezyx commented Jul 3, 2025

Add support for value packing, a change in protocol introduced in https://docs.datadoghq.com/developers/dogstatsd/datagram_shell/?tab=metrics#dogstatsd-protocol-v11

  • For counters, sum all values in single metric line
  • For gauge, take the last value assuming it was the last that arrived
  • For timers, either append all elements to existing array or use new one

for _, v := range m.Values {
valueSum += v
}
value := int64(valueSum / m.Rate)
Copy link
Contributor Author

@Wheezyx Wheezyx Jul 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this one but sum / rate is the same as sum (each / rate) so it should be fine

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It puts it on the sender to include rate as part of the consolidation key (which I know Jira has considered), but no hazard on the server side.

@Wheezyx Wheezyx marked this pull request as ready for review July 4, 2025 12:54
@tiedotguy tiedotguy requested review from hstan, irisgve, mattpatl and tiedotguy and removed request for hstan and irisgve July 4, 2025 14:36
tiedotguy
tiedotguy previously approved these changes Jul 4, 2025
Copy link
Collaborator

@tiedotguy tiedotguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

return ms[i].StringValue < ms[j].StringValue
} else {
return ms[i].Value < ms[j].Value
return ms[i].Values[0] < ms[j].Values[0]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for other reviewers: this code path is used in tests, to provide stable sorting, and making it easier to compare lists. It doesn't need to be perfect in production, just good enough in tests.

func (mm *MetricMap) receiveGauge(m *Metric, tagsKey string) {
v, ok := mm.Gauges[m.Name]
// Assuming here that the last value in value packing chain came as last
// It might be incorrect if sender sorts the array before sending but why would they
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:kekw:

for _, v := range m.Values {
valueSum += v
}
value := int64(valueSum / m.Rate)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It puts it on the sender to include rate as part of the consolidation key (which I know Jira has considered), but no hazard on the server side.

v, err := strconv.ParseFloat(l.m.StringValue, 64)
if err != nil {
return nil, nil, err
// Count number of colons to preallocate array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it count number of colons? why it starts from 1?
based on the code below I guess it's actually the count of values split by colons?

Copy link
Collaborator

@tiedotguy tiedotguy Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We start at count = 1. There's always 1 more value than there is colons.

edit: unless there's empty values, but this is pre-alocation, so we're just trying to avoid re-allocations. We can go a bit over.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the comment and logic a bit to make it clearer.

@tiedotguy tiedotguy merged commit f5fa12a into atlassian:master Jul 28, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants