Skip to content

Commit 05d910a

Browse files
committed
fix(bigtable): preserve defaultCluster sentinel in ingestMetadata on missing headers
ingestMetadata gated the entire extractLocation stamping block on err == nil. extractLocation returns (defaultCluster, defaultZone, err) when the LocationMDKey header is absent, and RecordAttemptCompletion's connectivity_error_count check reads isLocationEmpty := currAttempt.clusterID == defaultCluster as the "no server-side signals" marker. With the gate, an attempt without location headers left clusterID as "" (fresh AttemptTracer zero value); the check evaluated "" == "<unspecified>" → false, and the counter never incremented. TestConnectivityErrorCount caught this: metrics_test.go:592: Metric connectivity_error_count for status Unavailable: got cumulative value 0, want 1 Fix drops the err == nil gate around the stamping block so the sentinel is applied on missing-header attempts (matching the pre-race- fix behaviour), and only sets locationExtracted = true when err == nil so a later InTrailer carrying real data can still overwrite the sentinel on the same attempt. Verified: go test -run TestConnectivityErrorCount -count=1 ./bigtable/ → pass go test -race -count=3 -run TestHandleRPC_ConcurrentInTrailerAndEnd_NoRace ./bigtable/internal/metrics/ → pass go test -race -count=1 ./bigtable/internal/metrics/... → pass
1 parent bb8b285 commit 05d910a

1 file changed

Lines changed: 28 additions & 21 deletions

File tree

bigtable/internal/metrics/tracer.go

Lines changed: 28 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -555,29 +555,36 @@ func (mt *Tracer) ingestMetadata(headerMD, trailerMD metadata.MD) {
555555
a := &mt.currOp.currAttempt
556556

557557
if !a.locationExtracted {
558-
if clusterID, zoneID, err := extractLocation(headerMD, trailerMD); err == nil {
559-
// Don't overwrite a cluster/zone that the vRPC path has
560-
// already populated (SessionTable sets these directly from
561-
// the ClusterInformation payload); only fill in if the
562-
// attempt's value is missing or the sentinel default.
563-
// lastClusterID / lastZoneID always track the freshest real
564-
// value so operation-level metrics get a sensible fallback.
565-
if clusterID != "" {
566-
if existing := a.clusterID; existing == "" || existing == defaultCluster {
567-
a.SetClusterID(clusterID)
568-
}
569-
if clusterID != defaultCluster {
570-
mt.currOp.lastClusterID = clusterID
571-
}
558+
// extractLocation returns (defaultCluster, defaultZone, err) when the
559+
// LocationMDKey header is missing. Stamp the sentinel anyway —
560+
// RecordAttemptCompletion uses clusterID == defaultCluster as the
561+
// "no server-side signals" marker that feeds connectivity_error_count.
562+
// Only mark locationExtracted when we got a real value, so a later
563+
// InTrailer carrying real data can still overwrite the sentinel.
564+
clusterID, zoneID, err := extractLocation(headerMD, trailerMD)
565+
// Don't overwrite a cluster/zone that the vRPC path has
566+
// already populated (SessionTable sets these directly from
567+
// the ClusterInformation payload); only fill in if the
568+
// attempt's value is missing or the sentinel default.
569+
// lastClusterID / lastZoneID always track the freshest real
570+
// value so operation-level metrics get a sensible fallback.
571+
if clusterID != "" {
572+
if existing := a.clusterID; existing == "" || existing == defaultCluster {
573+
a.SetClusterID(clusterID)
572574
}
573-
if zoneID != "" {
574-
if existing := a.zoneID; existing == "" || existing == defaultZone {
575-
a.SetZoneID(zoneID)
576-
}
577-
if zoneID != defaultZone {
578-
mt.currOp.lastZoneID = zoneID
579-
}
575+
if clusterID != defaultCluster {
576+
mt.currOp.lastClusterID = clusterID
580577
}
578+
}
579+
if zoneID != "" {
580+
if existing := a.zoneID; existing == "" || existing == defaultZone {
581+
a.SetZoneID(zoneID)
582+
}
583+
if zoneID != defaultZone {
584+
mt.currOp.lastZoneID = zoneID
585+
}
586+
}
587+
if err == nil {
581588
a.locationExtracted = true
582589
}
583590
}

0 commit comments

Comments
 (0)