Skip to content

[ENHANCEMENT]: Add observability metrics to SetStatus, GetStatus, and GetTokenRequest in auditor.Service #1674

Description

@Rama542

Problem

The auditor.Service in token/services/auditor/auditor.go instruments its
write operations with Prometheus metrics — Audit(), Append(), and Release()
all record duration histograms and error counters. But the three query
methods SetStatus, GetStatus, and GetTokenRequest have zero instrumentation.

In production these are the most frequently called methods on the audit
path every finality listener update goes through SetStatus, and every
audit query goes through GetStatus and GetTokenRequest. Without metrics,
operators have no way to detect slow database queries, alert on elevated
error rates, or observe p99 latency on these critical paths. The blind spot
makes it much harder to debug audit issues in a live network.

Solution

Add duration histograms and error counters for the three uninstrumented
methods, following the exact same pattern already used in the same file:

In metrics.go add to the Metrics struct:
SetStatusDuration metrics.Histogram
SetStatusErrors metrics.Counter
GetStatusDuration metrics.Histogram
GetStatusErrors metrics.Counter
GetTokenRequestDuration metrics.Histogram
GetTokenRequestErrors metrics.Counter

In auditor.go wire them at each call site:
func (a *Service) SetStatus(ctx context.Context, ...) error {
start := time.Now()
defer func() { a.metrics.SetStatusDuration.Observe(time.Since(start).Seconds()) }()
if err := a.auditDB.SetStatus(ctx, txID, status, message); err != nil {
a.metrics.SetStatusErrors.Add(1)
return err
}
return nil
}

Same pattern for GetStatus and GetTokenRequest.
This is purely additive no existing behaviour changes.

Alternatives

I considered adding a single catch-all middleware wrapper around auditDB
calls, but that would require changes across multiple layers. The per-method
approach is simpler, more explicit, and consistent with how Audit() and
Append() are already instrumented in the same file.

Additional Context

Affected files:
token/services/auditor/auditor.go SetStatus:189, GetStatus:195, GetTokenRequest:200
token/services/auditor/metrics.go Metrics struct and newMetrics factory

The full infrastructure is already in place:

  • metrics.Provider is injected via NewService()
  • noopProvider handles the case where no provider is configured
  • Histogram bucket configuration already defined for similar methods

This requires no new dependencies and no interface changes.
I am happy to open a PR for this if the team agrees with the approach.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Fields

No fields configured for Feature.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions