Skip to content

Fix "metric was collected before" error #433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

colinleroy
Copy link

Hello,

Since one or two weeks, our stackdriver_exporter errors out with logs similar to:

collected metric "stackdriver_network_security_policy_networksecurity_googleapis_com_https_request_count" { label:{name:"backend_target_name"  value:"****-https-lb-spectrum-back"}  label:{name:"blocked"  value:"true"}  label:{name:"location"  value:"global"}  label:{name:"policy_name"  value:"preconfigured-rules-policy"}  label:{name:"project_id"  value:"****-production"}  label:{name:"unit"  value:"1"}  gauge:{value:0}  timestamp_ms:1746012857310} was collected before with the same name and label values

After investigation, it comes from the fact that some of these metrics are returned twice by Stackdriver.

This PR adds a filtering to reportTimeSeriesMetrics() in order to avoid sending duplicate time series to the Prometheus Collector.

I hope this helps and I hope it's good enough, I'm not very well-versed in Go.

Thanks!

Signed-Off-By: Colin Leroy-Mira <[email protected]>
The networksecurity.googleapis.com/https/request_count
sometimes returns duplicate series. Filter them out to
avoid errors.

Signed-Off-By: Colin Leroy-Mira <[email protected]>
@aarozhkov
Copy link

@colinleroy
Stack on this too.
Try to use binary builded from your PR.

Lost other valid metrics.
Example stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_ack_latencies_bucket
I have a lot of subscribtion_ids.
Left with only on subscription metric.

@colinleroy
Copy link
Author

Were they metrics with at least one unique key/value label ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants