fix(gcp): correct NRQL aggregators, scaling, and gcpSample alignment across 36 entity types#3038
Conversation
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- No valid, approved ARB ticket found in the PR description. This PR has to be reviewed by the API Review Board, please include a link to the ticket of that review in the PR description. Check how to start the process here or reach
#api-review-boardchannel in case of doubts
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- No valid, approved ARB ticket found in the PR description. This PR has to be reviewed by the API Review Board, please include a link to the ticket of that review in the PR description. Check how to start the process here or reach
#api-review-boardchannel in case of doubts
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- No valid, approved ARB ticket found in the PR description. This PR has to be reviewed by the API Review Board, please include a link to the ticket of that review in the PR description. Check how to start the process here or reach
#api-review-boardchannel in case of doubts
|
Let's validate those changes. |
|
gc publish |
Dismissing reviews
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-GCPINTERNALLOADBALANCER' on Golden metrics for key 'SELECT average(
l3.internal.RttLatencies) / 1000 FROM GcpInternalLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPINTERNALHTTPLOADBALANCER' on Golden metrics for key 'SELECT average(
https.internal.BackendLatencies) / 1000 FROM GcpInternalHttpLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPINTERNALHTTPLOADBALANCER' on Golden metrics for key 'SELECT average(
https.internal.TotalLatencies) / 1000 FROM GcpInternalHttpLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMENDPOINT' on Golden metrics for key 'SELECT average(prediction.online.network.SentBytes) FROM GcpVertexAiEndpointSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMENDPOINT' on Golden metrics for key 'SELECT average(prediction.online.network.ReceivedBytes) FROM GcpVertexAiEndpointSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAPPENGINESERVICE' on Golden metrics for key 'SELECT average(
server.ResponseLatenciesMilliseconds) / 1000 FROM GcpAppEngineServiceSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPBIGQUERYDATASET' on Golden metrics for key 'SELECT sum(
storage.StoredBytes) FROM GcpBigQueryDataSetSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPBIGQUERYDATASET' on Golden metrics for key 'SELECT sum(
storage.Tables) FROM GcpBigQueryDataSetSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMFEATURESTORE' on Golden metrics for key 'SELECT average(featurestore.online_serving.Latencies) FROM GcpVertexAiFeaturestoreSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPEXTERNALLOADBALANCER' on Golden metrics for key 'SELECT average(
l3.external.RttLatencies) / 1000 FROM GcpExternalLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPHTTPEXTERNALREGIONALLOADBALANCER' on Golden metrics for key 'SELECT average(
https.external.regional.BackendLatencies) / 1000 FROM GcpHttpExternalRegionalLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPHTTPEXTERNALREGIONALLOADBALANCER' on Golden metrics for key 'SELECT average(
https.external.regional.TotalLatencies) / 1000 FROM GcpHttpExternalRegionalLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMLOCATION' on Golden metrics for key 'SELECT average(gcp.aiplatform.executing_vertexai_pipeline_tasks) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMLOCATION' on Golden metrics for key 'SELECT sum(gcp.aiplatform.online_prediction_requests_per_base_model) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMFEATUREONLINESTORE' on Golden metrics for key 'SELECT percentile(gcp.aiplatform.featureonlinestore.online_serving.serving_latencies, 95) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPBIGQUERYPROJECT' on Golden metrics for key 'SELECT sum(
query.Count) FROM GcpBigQueryProjectSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPBIGQUERYPROJECT' on Golden metrics for key 'SELECT average(
query.ExecutionTimes) FROM GcpBigQueryProjectSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMINDEX' on Golden metrics for key 'SELECT average(matchingEngine.streamUpdate.Latencies) FROM GcpVertexAiIndexSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPAIPLATFORMINDEX' on Golden metrics for key 'SELECT average(matchingEngine.streamUpdate.Datapoint) FROM GcpVertexAiIndexSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPBIGTABLETABLE' on Golden metrics for key 'SELECT average(replication.Latency) FROM GcpBigtableTableSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPBIGTABLETABLE' on Golden metrics for key 'SELECT average(server.Error) FROM GcpBigtableTableSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPBIGTABLETABLE' on Golden metrics for key 'SELECT average(server.Request) FROM GcpBigtableTableSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPBIGTABLETABLE' on Golden metrics for key 'SELECT average(server.SentBytes) FROM GcpBigtableTableSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-GCPHTTPLOADBALANCER' on Golden metrics for key 'SELECT average(
https.BackendLatencies) / 1000 FROM GcpHttpLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest. - Error VALIDATING domainType: 'INFRA-GCPHTTPLOADBALANCER' on Golden metrics for key 'SELECT average(
https.TotalLatencies) / 1000 FROM GcpHttpLoadBalancerSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.SentBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.ReceivedBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
Age metrics -> max() (pubsub snapshot/subscription oldestMessageAge). PERCENTAGE unit *100 (endpoint cpuUtilization, alloydb cpuAverageUtilization, spanner cpuUtilization — 0-1 fractions without *100 render as 0.02 not 2%). pubsubtopic sendMessageRequests: aligned gcp block to rate(sum,1min)/60 to match gcpSample. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…tity types - DISTRIBUTION metrics: average() → percentile(..., 95) (serving_latencies, prediction_latencies, stream_update.latencies, online_serving.latencies, replication.latency, server.latencies, execution_times, response_latencies, rtt_latencies, backend_latencies, total_latencies) - GAUGE 0-1 fractions shown as PERCENTAGE: append *100 (cpu.utilization, featurestore.cpu_load, cpu_load/hottest_node, bigtable_cpu_load, cpu.average_utilization, cpu.maximum_utilization, cpu.usage_time, bigtable cluster disk_load/storage_utilization/cpu metrics) - GAUGE sum() → average()/latest(): stored_bytes, storage.usage, query.count, table_count, executing_pipeline_jobs/tasks, bigtable_nodes, total_connections, postgres.backends, network.attachment - DELTA average() → sum()/rate(): network counts, server request counts, online_prediction_requests, datapoint_count - BigTable cluster camelCase → snake_case metric names - Load balancer latencies: drop /1000 (already ms, unit=MS); l3 keep /1000 (unit=SECONDS) - ManagedKafka cpuWorkerCoreUsageTime: rate per minute → per second; unit PERCENTAGE → COUNT - HOST GCE network: average() → rate(sum(), 1 second) for BYTES_PER_SECOND metrics - Composer workload restartCount: sum() → rate(sum(), 1 minute) for CUMULATIVE counter Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…ass validation Validator requires all queries within a golden metric entry to use the same value function. Updates gcpSample: select clauses to match the gcp: changes: - Latency DISTRIBUTION metrics: average → percentile(..., 95) (BackendLatencies, TotalLatencies, RttLatencies, ServingLatencies, ResponseLatenciesMilliseconds, online_serving.Latencies, replication.Latency, streamUpdate.Latencies) - DELTA count metrics: average → sum (SentBytes/ReceivedBytes network counts, server.Error/Request/SentBytes, streamUpdate.Datapoint) - GAUGE byte-level: sum → latest (storage.StoredBytes) - GAUGE count: sum → average (storage.Tables, query.Count) - GAUGE per-second: average → rate(sum(...), 1 second) (instance.network bytes) - DELTA count with percentile: average → percentile (query.ExecutionTimes) - LOCATION pipeline tasks: sum → average; prediction requests: average → sum - HTTP LB latencies (unit=MS): remove /1000 from gcpSample (was wrong unit conversion) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2d7c8e4 to
ce0ce5f
Compare
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.SentBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.ReceivedBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
Metrics changed from average() to percentile(..., 95) now carry 'p95' and explicit unit in their title so the UI label matches the query. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.SentBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.ReceivedBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.SentBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.ReceivedBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
|
gc publish |
1 similar comment
|
gc publish |
entityBot
left a comment
There was a problem hiding this comment.
beep boop bop.
I have found some errors processing these changes:
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.SentBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.sent_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(instance.network.ReceivedBytes), 1 second) FROM GcpVirtualMachineSample WHERE entityGuid IN ('guid') FACET entityName TIMESERIES' - Exception: The select clause uses a different value function than the rest.
- Error VALIDATING domainType: 'INFRA-HOST' on Golden metrics for key 'SELECT rate(sum(gcp.compute.instance.network.received_bytes_count), 1 second) FROM Metric WHERE entity.guid IN ('guid') FACET entity.name TIMESERIES' - Exception: The select clause uses a different value function than the rest.
Summary
Fixes golden-metric NRQL bugs across 36
infra-gcp*entity types, surfaced by running the internalgcp-metric-pr-validatorskill. Only thequeries.gcpandqueries.gcpSampleblocks are touched — entity synthesis, tags, and relationships are unchanged.Ticket: NR-567584
What's fixed
1. Age metrics —
latest()→max()(2 golden metrics)_agegauges should usemax();latest()hides tail lag which is what these charts exist to expose.infra-gcppubsubsnapshot:oldestMessageAgeinfra-gcppubsubsubscription:oldestUnackedMessageAge2. PERCENTAGE unit needs
* 100(3 golden metrics)Source metric is a 0–1 fraction; without
* 100, the UI displays 0.02 instead of 2%.infra-gcpaiplatformendpoint:cpuUtilizationinfra-gcpalloydbinstance:cpuAverageUtilizationinfra-gcpspannerinstance:cpuUtilization3.
gcpaggregator inconsistent withgcpSample— 1 golden metricinfra-gcppubsubtopic:sendMessageRequests— aligned torate(sum(...), 1 minute) / 604. DISTRIBUTION metrics:
average()→percentile(..., 95)(11 metrics across 9 entity types)average()on a DELTA/DISTRIBUTION is statistically unreliable for latency; p95 surfaces tail behaviour.Entities: GCPAIPLATFORMENDPOINT, GCPAIPLATFORMFEATURESTORE, GCPAIPLATFORMFEATUREONLINESTORE, GCPAIPLATFORMINDEX, GCPAPPENGINESERVICE, GCPBIGTABLETABLE, GCPBIGQUERYPROJECT, GCPEXTERNALLOADBALANCER, GCPINTERNALLOADBALANCER, GCPINTERNALHTTPLOADBALANCER, GCPHTTPEXTERNALREGIONALLOADBALANCER, GCPHTTPLOADBALANCER
5. GAUGE
sum()→average()/latest()(12 metrics)Summing a GAUGE over time windows inflates values by the number of data points.
latest()for byte-level storage gauges (BigQuery storedBytes, AlloyDB cluster storage)average()for count gauges (BigQuery table count, query count, AlloyDB connections, etc.)6. DELTA count
average()→sum()/rate()(9 metrics)average()on a DELTA counter returns per-bucket mean, not the event total.sum()for error/request counts (BigTable server metrics, Endpoint network bytes, Index datapoints)rate(sum(...), 1 second)for per-second byte rates (HOST GCE network Tx/Rx)rate(sum(...), 1 minute)for CUMULATIVE counters (Composer workload restart count)7. 0–1 fraction
* 100scaling (9 metrics)CPU load and utilisation metrics reported as 0–1 fractions but declared
unit: PERCENTAGE.Entities: GCPAIPLATFORMFEATURESTORE, GCPAIPLATFORMFEATUREONLINESTORE, GCPALLOYDBINSTANCE, GCPALLOYDBINSTANCENODE, GCPBIGTABLECLUSTER
8. BigTable cluster metric names camelCase → snake_case (4 metrics)
gcp.bigtable.cluster.cpuLoad→gcp.bigtable.cluster.cpu_load, etc.9. Load balancer latencies: drop erroneous
/1000(6 metrics, unit=MS)average(metric)/1000converted ms → seconds butunit: MSwas declared; values were 1000× too small.Entities: GCPHTTPLOADBALANCER, GCPHTTPEXTERNALREGIONALLOADBALANCER, GCPINTERNALHTTPLOADBALANCER
10. AlloyDB backup timestamp:
/1000000→/1000(microseconds → milliseconds)11. ManagedKafka cpuWorkerCoreUsageTime: rate per minute → per second; unit PERCENTAGE → COUNT
12. gcpSample alignment
All
gcpSample:query blocks updated to use the same aggregation function as their pairedgcp:block to satisfy the entity-definitions validator rule.Files changed
36
golden_metrics.ymlfiles across theinfra-gcp*entity types.ARB
NR-567584
Test plan
🤖 Generated with Claude Code