docs: document exported metric names and coverage gaps

The SDK never spells out the metric names Prometheus exports. The
fully-qualified name is assembled at registration time from the Go package
of the caller, and the TMS-scoped provider in token/core/common/metrics adds
a stack frame, so metrics created through it are exported under that
package rather than their own. Documentation written from the bare Name
field of the options therefore lists names no query matches.

Add docs/development/metrics.md: the derivation rules, every metric under
the name Prometheus actually serves, its type, labels, meaning and source,
example queries, and a ranked analysis of what a node still cannot report.

Guard the page with token/services/metricsdoc, which instantiates every
metrics constructor the way production wires it, reads the resulting names
back out of a Prometheus registry, and compares them against a golden file
and against the page itself. Renaming, adding or relocating a metric fails
the test until the documentation is updated.

Export NewMetrics/NewClientMetrics in the auditor, certification and
Fabric-X queue packages so the guard can build their instrumentation
without constructing the surrounding service.

Correct docs/drivers/metrics.md, which listed declared names as if they
were queryable, and make docs/development/monitoring.md the entry point.

Signed-off-by: Hayim.Shaul@ibm.com <hayimsha@fhe03.vpc.cloud9.ibm.com>

Author: Hayim.Shaul@ibm.com

docs/README.md

@@ -30,6 +30,7 @@ If you are developing *using* Panurus or contributing *to* Panurus, check out th
 *   [General Guidelines](development/general.md)
 *   [Idiomatic Go](development/idiomatic.md)
 *   [Testing](development/testing.md)
+*   [Monitoring](development/monitoring.md) and the [Metrics Reference](development/metrics.md)
 *   [Test & Benchmark Profiler](../cmd/profiler/README.md)
 
 ## Evolution

docs/development/development.md

@@ -8,6 +8,7 @@ This page contains link to the development guidelines and more.
 - [Development Tools](./tools.md)
 - [Linting](./linting.md)
 - [Monitoring](./monitoring.md)
+- [Metrics Reference](./metrics.md)
 - [Mock Files Generation](./mock.md)
 - [AI Agents Best Practices](./ai_agents.md)
 - [Tools: tokengen](./tokengen.md)

docs/development/metrics.md

@@ -5,3 +5,13 @@ We adopt the monitoring infrastructure provided by the [`Fabric Smart Client`](h
 We use the following two methods to monitor the performance of the application:
 * **Metrics** provide an overview of the overall system performance using aggregated results, e.g. total requests, requests per second, current state of a variable, average duration, percentile of duration
 * **Traces** help us analyze single requests by breaking down their lifecycles into smaller components
+
+## Where to look next
+
+* [Metrics Reference](./metrics.md) — every metric Panurus exports, under the exact name Prometheus
+  serves it, plus how those names are derived, example queries, and the current coverage gaps.
+* [Driver Metrics](../drivers/metrics.md) — how the driver service wrappers are built and which
+  methods they instrument.
+* [Fabric Smart Client monitoring](https://github.com/hyperledger-labs/fabric-smart-client/blob/main/docs/platform/view/monitoring.md)
+  — the platform metrics and traces Panurus inherits (views, sessions, gRPC, process), and how to
+  enable the Prometheus endpoint and the tracing exporter.

docs/development/monitoring.md

@@ -20,6 +20,14 @@ implementation, and records three metrics per method invocation:
 | **Histogram** (`*_duration_seconds`) | Execution duration of each call |
 | **Counter** (`*_errors_total`) | Total number of calls that returned an error |
 
+The metric names in this page are the **declared** names, as written in the wrapper sources.
+Prometheus exports them under a prefix derived from the package that creates them; because the driver
+wrappers receive a TMS-scoped provider, every metric below is exported as
+`panurus_core_common_metrics_<declared name>` — for instance `issue_service_operations_total` is
+queried as `panurus_core_common_metrics_issue_service_operations_total`. See
+[Metrics Reference](../development/metrics.md) for the derivation rules and the exported names of
+every metric in the SDK.
+
 All metrics carry four labels for multi-TMS filtering:
 
 | Label | Description |
@@ -136,27 +144,12 @@ Metrics emitted:
 
 ## Metric Reference
 
-The full list of metrics emitted by the driver wrappers:
-
-| Metric Name | Type | Description |
-|-------------|------|-------------|
-| `issue_service_operations_total` | Counter | Total `IssueService` method invocations |
-| `issue_service_duration_seconds` | Histogram | Duration of `IssueService` method calls |
-| `issue_service_errors_total` | Counter | Total `IssueService` method errors |
-| `transfer_service_operations_total` | Counter | Total `TransferService` method invocations |
-| `transfer_service_duration_seconds` | Histogram | Duration of `TransferService` method calls |
-| `transfer_service_errors_total` | Counter | Total `TransferService` method errors |
-| `auditor_service_operations_total` | Counter | Total `AuditorService` method invocations |
-| `auditor_service_duration_seconds` | Histogram | Duration of `AuditorService` method calls |
-| `auditor_service_errors_total` | Counter | Total `AuditorService` method errors |
-| `tokens_service_operations_total` | Counter | Total `TokensService` method invocations |
-| `tokens_service_duration_seconds` | Histogram | Duration of `TokensService` method calls |
-| `tokens_service_errors_total` | Counter | Total `TokensService` method errors |
-| `tokens_upgrade_service_operations_total` | Counter | Total `TokensUpgradeService` method invocations |
-| `tokens_upgrade_service_duration_seconds` | Histogram | Duration of `TokensUpgradeService` method calls |
-| `tokens_upgrade_service_errors_total` | Counter | Total `TokensUpgradeService` method errors |
-
-All metrics use labels: `network`, `channel`, `namespace`, `method`.
+The exported names, types and labels of the fifteen driver metrics are listed in
+[Metrics Reference — Driver services](../development/metrics.md#driver-services). That page is kept in
+step with the code by `token/services/metricsdoc`, so it is the authoritative list; this page describes
+only how the wrappers work and which methods they instrument.
+
+All driver metrics use labels: `network`, `channel`, `namespace`, `method`.
 
 ## Source
 

docs/drivers/metrics.md

@@ -22,6 +22,7 @@ require (
 	github.com/jackc/pgxlisten v0.0.0-20250802141604-12b92425684c
 	github.com/onsi/ginkgo/v2 v2.32.0
 	github.com/onsi/gomega v1.42.1
+	github.com/prometheus/client_golang v1.24.1
 	github.com/stretchr/testify v1.11.1
 	github.com/tidwall/gjson v1.19.0
 	go.opentelemetry.io/otel/trace v1.44.0
@@ -100,7 +101,6 @@ require (
 	github.com/pelletier/go-toml/v2 v2.3.1 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
-	github.com/prometheus/client_golang v1.24.1 // indirect
 	github.com/prometheus/client_model v0.6.2 // indirect
 	github.com/prometheus/common v0.70.1 // indirect
 	github.com/prometheus/procfs v0.21.1 // indirect

go.mod

@@ -70,6 +70,14 @@ func newMetrics(p metrics.Provider) *Metrics {
 	}
 }
 
+// NewMetrics creates a new Metrics instance with the given provider.
+// It is exported so that the metrics reference guard can instantiate the
+// auditor instrumentation without building a full Service; see
+// docs/development/metrics.md.
+func NewMetrics(p metrics.Provider) *Metrics {
+	return newMetrics(p)
+}
+
 // noopProvider discards all observations. Used when no provider is configured.
 type noopProvider struct{}
 

token/services/auditor/metrics.go

@@ -68,6 +68,14 @@ type ClientMetrics struct {
 	DroppedTokens metrics.Counter
 }
 
+// NewClientMetrics creates a new ClientMetrics instance with the given provider.
+// It is exported so that the metrics reference guard can instantiate the
+// certification client instrumentation without building a full
+// CertificationClient; see docs/development/metrics.md.
+func NewClientMetrics(p metrics.Provider) *ClientMetrics {
+	return newClientMetrics(p)
+}
+
 func newClientMetrics(p metrics.Provider) *ClientMetrics {
 	return &ClientMetrics{
 		RequestDuration: p.NewHistogram(certificationRequestDuration),

token/services/certifier/interactive/metrics.go

@@ -0,0 +1,21 @@
+/*
+Copyright IBM Corp. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+// Package metricsdoc guards the metrics reference in docs/development/metrics.md.
+//
+// The SDK never spells out the metric names it exports: the fully-qualified
+// Prometheus name of a metric is assembled at registration time from the Go
+// package that creates it, so the same CounterOpts produce different exported
+// names depending on which package - and which provider wrapper - the metric
+// travels through. Documentation written from the bare Name field of the opts
+// therefore lists names that no Prometheus query will ever match.
+//
+// The test in this package instantiates every metrics constructor of the SDK
+// the way production wiring does, reads the resulting names back out of a
+// Prometheus registry, and compares them both against a golden file and against
+// the reference documentation. Adding, renaming or relocating a metric fails the
+// test until the documentation is updated.
+package metricsdoc