OpenTelemetry metrics support via vendor-neutral ydb.observability

Add client-side metrics on the ydb.observability layer, symmetric with tracing,
with the metric machinery kept out of the query-service code.

- ydb/observability/metrics.py: the vendor-neutral MetricsProvider Protocol
  (record / add / observe_gauge — one method per instrument kind), a Noop default,
  and the facades that hide all bookkeeping — SessionMetrics, QuerySessionPoolMetrics
  (with conditional timing context managers), and the observe_retry_metrics decorator.
  The asynchronous-gauge state (open sessions per pool, pool size) is owned here, so
  backends only implement three generic methods. Enabling metrics adds a
  ydb-sdk-metrics/0.1.0 token via the sdk_build_info_tokens aggregator.
- ydb/observability/tracing.py: create_ydb_span returns a span-compatible composite
  that drives tracing and metrics for one operation (zero overhead when both off).
- ydb/opentelemetry/metrics_plugin.py: OtelMetricsProvider installed through the
  observability layer; ydb.opentelemetry.enable_metrics(meter_provider=None)
  convenience and a lazily-exposed OtelMetricsProvider. The SDK core never imports
  opentelemetry.
- Pool/session/retry code carries only clean facade calls — no time.monotonic,
  attribute dicts, or record_* scattered inline. QuerySessionPool gains an optional
  name for the pool metric label.
- Consolidate observability tests under tests/observability/; document metrics and a
  custom-backend example on the observability + opentelemetry pages; extend the OTel
  example. Fix a concurrent patch.object leak in the async tracing tests and allow the
  example scripts into the Docker build context (.dockerignore).

Author: vgvoleg

.dockerignore

@@ -4,4 +4,7 @@
 !README.md
 !requirements.txt
 !pyproject.toml
-!setup.py
+!setup.py
+!examples/opentelemetry/requirements.txt
+!examples/opentelemetry/otel_example.py
+!examples/opentelemetry/load_tank.py

CHANGELOG.md

@@ -1,5 +1,6 @@
 * Introduce `ydb.observability` — vendor-neutral tracing entrypoint with a `TracingProvider` interface; `enable_tracing(provider)` accepts any implementation (custom or OpenTelemetry) and replaces the previously installed one. The SDK core no longer imports `opentelemetry`, so tracing can be enabled without the OpenTelemetry packages by supplying a custom provider
 * When tracing is enabled the SDK appends a `ydb-sdk-tracing/0.1.0` token to the `x-ydb-sdk-build-info` header, so the server can distinguish requests from tracing-enabled clients
+* Add client-side metrics through the same vendor-neutral `ydb.observability` layer: `enable_metrics(provider)` / `disable_metrics()`, with `ydb.opentelemetry.enable_metrics(meter_provider=None)` as the OpenTelemetry convenience. Instruments cover client operation duration/failures, retry duration/attempts, and query session pool state. `QuerySessionPool` gains an optional `name` argument for the pool metric label. When metrics are enabled the SDK appends a `ydb-sdk-metrics/0.1.0` token to the `x-ydb-sdk-build-info` header
 
 ## 3.30.0 ##
 * Query session attach stream handles `NodeShutdown` and `SessionShutdown` session hints: on `NodeShutdown` the session's node connection is pessimized and the session is retired, on `SessionShutdown` the session is retired without touching the node

docs/observability.rst

@@ -8,9 +8,14 @@ driver initialization, and retries — are turned into spans and handed to which
 built-in backend (see :doc:`opentelemetry`), but any tracing system can be plugged in
 by implementing the interface described below.
 
-Tracing is **zero-cost when disabled**: until you install a provider every span is a
-no-op stub, and the SDK never imports ``opentelemetry`` — or any other backend — on its
-own. The dependency is pulled in only when you explicitly opt into a concrete provider.
+The same layer also exposes **client-side metrics** (operation latency and failures,
+retry cost, query session pool state) through :func:`ydb.observability.enable_metrics`.
+Tracing and metrics are independent — enable either, both, or neither.
+
+Observability is **zero-cost when disabled**: until you install a backend every span is a
+no-op stub and every metric is dropped by a no-op provider, and the SDK never imports
+``opentelemetry`` — or any other backend — on its own. The dependency is pulled in only
+when you explicitly opt into a concrete backend.
 
 
 The Tracing Interface
@@ -247,3 +252,163 @@ A custom ``attach_context`` must therefore honour this three-way contract on blo
 
 The ``LoggingSpan`` above already implements exactly this. Getting it wrong leaks spans
 (never ended) or ends them too early (streaming spans that miss the actual read time).
+
+
+Metrics
+-------
+
+Client-side metrics are enabled independently of tracing:
+
+.. code-block:: python
+
+    from ydb.observability import enable_metrics, disable_metrics
+
+    enable_metrics(provider)   # install a metrics backend
+    disable_metrics()          # turn metrics off — back to the no-op default
+
+For OpenTelemetry the built-in convenience is ``ydb.opentelemetry.enable_metrics``,
+which builds an OTel-backed provider from a meter provider and installs it here — see
+the :doc:`opentelemetry` page for the meter-provider and exporter setup.
+
+The Metrics Interface
+~~~~~~~~~~~~~~~~~~~~~~
+
+A metrics backend implements :class:`~ydb.observability.MetricsProvider` — three
+methods, one per instrument kind:
+
+.. code-block:: python
+
+    class MetricsProvider(Protocol):
+        def record(self, name, value, attributes=None): ...       # value distribution (histogram)
+        def add(self, name, value, attributes=None): ...          # additive counter (may go negative)
+        def observe_gauge(self, name, callback): ...              # asynchronous gauge
+
+``record`` and ``add`` are pushed at the moment of the event. ``observe_gauge`` is
+different: the SDK owns the accumulated state (open sessions per pool, configured pool
+size) and hands the backend a ``callback`` that returns the current
+``(value, attributes)`` observations; the backend reads it whenever it collects. This
+keeps the pool bookkeeping vendor-neutral instead of pushing it into every backend.
+
+It is a :class:`typing.Protocol`, so a backend does **not** need to subclass anything —
+any object with these three methods works.
+
+Instruments
+~~~~~~~~~~~
+
+The SDK records the following instruments regardless of backend (the OpenTelemetry
+adapter maps them to instruments on the ``"ydb.sdk"`` meter):
+
+.. list-table::
+   :header-rows: 1
+   :widths: 32 22 46
+
+   * - Metric
+     - Kind
+     - Description
+   * - ``db.client.operation.duration``
+     - Histogram (``s``)
+     - Latency of user-visible YDB client operations.
+   * - ``ydb.client.operation.failed``
+     - Counter
+     - Failed user-visible YDB client operations.
+   * - ``ydb.query.session.create_time``
+     - Histogram (``s``)
+     - Time spent creating a query session.
+   * - ``ydb.query.session.pending_requests``
+     - UpDownCounter
+     - Requests currently waiting for a session from the pool.
+   * - ``ydb.query.session.timeouts``
+     - Counter
+     - Session acquisition timeouts.
+   * - ``ydb.query.session.count``
+     - ObservableUpDownCounter
+     - Current number of open query sessions by pool and ``ydb.query.session.state`` (``idle`` / ``used``).
+   * - ``ydb.query.session.max``
+     - ObservableUpDownCounter
+     - Maximum configured number of sessions for a query session pool.
+   * - ``ydb.query.session.min``
+     - ObservableUpDownCounter
+     - Minimum configured pool size. The SDK sets no minimum, so this is always ``0``.
+   * - ``ydb.client.retry.duration``
+     - Histogram (``s``)
+     - Total duration of a logical retried operation, including attempts and backoff.
+   * - ``ydb.client.retry.attempts``
+     - Histogram
+     - Number of attempts performed for one logical retried operation.
+
+Attributes
+~~~~~~~~~~
+
+Operation metrics (``db.client.operation.*``) carry stable labels only:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 35 65
+
+   * - Attribute
+     - Description
+   * - ``database``
+     - Database path.
+   * - ``endpoint``
+     - Configured endpoint in ``host:port`` form.
+   * - ``operation.name``
+     - SDK operation name without the ``ydb.`` prefix, e.g. ``"ExecuteQuery"``.
+   * - ``status_code``
+     - Added only to ``ydb.client.operation.failed``.
+
+Operation metrics are recorded for ``ExecuteQuery``, ``Commit``, ``Rollback``,
+``CreateSession`` and ``BeginTransaction``. Query session metrics use
+``ydb.query.session.pool.name``: when ``name`` is not passed to ``QuerySessionPool``
+the SDK falls back to the driver connection string ``<endpoint><database>`` (e.g.
+``grpc://localhost:2136/local``) so the pool is identifiable out of the box. Pass
+``QuerySessionPool(..., name="main-pool")`` (sync or async) when several pools share a
+connection string. Retry metrics are recorded without attributes.
+
+Writing a Custom Metrics Backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To send metrics somewhere OpenTelemetry does not cover — a home-grown collector, a log
+line, an in-memory recorder for tests — implement the three methods. Nothing to
+subclass; just match the signatures:
+
+.. code-block:: python
+
+    from ydb.observability import enable_metrics, disable_metrics
+
+
+    class LoggingMetrics:
+        def __init__(self):
+            self._gauges = {}
+
+        def record(self, name, value, attributes=None):
+            # A value distribution (histogram): operation/create/retry durations, attempts.
+            print(f"[metric] {name} = {value} {attributes or {}}")
+
+        def add(self, name, value, attributes=None):
+            # An additive counter; value may be negative (pending_requests goes up and down).
+            print(f"[metric] {name} += {value} {attributes or {}}")
+
+        def observe_gauge(self, name, callback):
+            # Asynchronous gauge: keep the callback and poll it whenever you collect.
+            # callback() -> iterable of (value, attributes).
+            self._gauges[name] = callback
+
+        def collect(self):
+            for name, callback in self._gauges.items():
+                for value, attributes in callback():
+                    print(f"[gauge] {name} = {value} {attributes}")
+
+
+    enable_metrics(LoggingMetrics())
+    # ... use the SDK; record()/add() fire as operations happen ...
+    disable_metrics()
+
+The instrument kind for each metric is implied by which method the SDK calls: names in
+the *Histogram* rows above arrive through ``record``, *Counter* / *UpDownCounter* names
+through ``add``, and the three *ObservableUpDownCounter* gauges
+(``ydb.query.session.count`` / ``max`` / ``min``) through ``observe_gauge``. A backend
+that has no notion of asynchronous gauges can simply store the callbacks (or ignore
+them); the SDK never pushes those values, so nothing is lost elsewhere.
+
+``disable_metrics()`` clears the SDK-side gauge state and reverts to the no-op default,
+so recording calls become cheap no-ops again.

docs/opentelemetry.rst

@@ -2,14 +2,17 @@ OpenTelemetry
 =============
 
 `OpenTelemetry <https://opentelemetry.io/>`_ is the built-in backend for the SDK's
-vendor-neutral tracing interface (see :doc:`observability`). Enabling it turns key YDB
-operations — session creation, query execution, transaction commit/rollback, driver
-initialization, and retries — into OpenTelemetry spans, and propagates trace context to
-the YDB server through gRPC metadata using the
-`W3C Trace Context <https://www.w3.org/TR/trace-context/>`_ standard.
+vendor-neutral observability interface (see :doc:`observability`). Enabling tracing turns
+key YDB operations — session creation, query execution, transaction commit/rollback,
+driver initialization, and retries — into OpenTelemetry spans, and propagates trace
+context to the YDB server through gRPC metadata using the
+`W3C Trace Context <https://www.w3.org/TR/trace-context/>`_ standard. Enabling metrics
+records client-side OpenTelemetry instruments for the same operations, retries, and
+query session pools.
 
 Like every backend, it is **zero-cost when disabled**: the SDK uses no-op stubs by
-default and does not import ``opentelemetry`` until you call ``enable_tracing()``.
+default and does not import ``opentelemetry`` until you call ``enable_tracing()`` or
+``enable_metrics()``.
 
 
 Installation
@@ -85,6 +88,52 @@ vendor-neutral entrypoint — this is exactly what the convenience wrapper above
     enable_tracing(OtelTracingProvider())          # or OtelTracingProvider(my_tracer)
 
 
+Enabling Metrics
+----------------
+
+Metrics are independent from tracing — enable either or both. Call ``enable_metrics()``
+once, after configuring your OpenTelemetry meter provider and before creating drivers or
+query session pools:
+
+.. code-block:: python
+
+    from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
+    from opentelemetry.sdk.metrics import MeterProvider
+    from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
+    from opentelemetry.sdk.resources import Resource
+
+    import ydb
+    from ydb.opentelemetry import enable_metrics
+
+    # 1. Set up OpenTelemetry
+    resource = Resource(attributes={"service.name": "my-service"})
+    metric_reader = PeriodicExportingMetricReader(
+        OTLPMetricExporter(endpoint="http://localhost:4317"),
+        export_interval_millis=1000,
+    )
+    meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
+
+    # 2. Enable YDB metrics
+    enable_metrics(meter_provider)
+
+    # 3. Use the SDK as usual — metrics are recorded automatically
+    with ydb.Driver(endpoint="grpc://localhost:2136", database="/local") as driver:
+        driver.wait(timeout=5)
+        with ydb.QuerySessionPool(driver, name="main-pool") as pool:
+            pool.execute_with_retries("SELECT 1")
+
+    meter_provider.shutdown()
+
+``enable_metrics()`` accepts an optional ``meter_provider`` argument. If omitted, the SDK
+obtains a meter named ``"ydb.sdk"`` from the global meter provider. The call is
+idempotent: repeated ``enable_metrics()`` calls do nothing until you call
+``disable_metrics()``, which clears the in-memory observable metric values and restores
+the no-op provider so metric recording stays a cheap no-op.
+
+The full list of instruments and their attributes is catalogued on the
+:doc:`observability` page.
+
+
 Instrumented Spans and Attributes
 ---------------------------------
 

examples/opentelemetry/Dockerfile

@@ -1,11 +1,13 @@
-# Isolated image for the OpenTelemetry demo. Build context is the repository root.
+# Isolated image for the OpenTelemetry demo scripts. Build context is the repository root.
 #
-#   docker compose -f examples/opentelemetry/compose-e2e.yaml build otel-example
+#   docker compose -f examples/opentelemetry/compose-e2e.yaml build
 #
 # A separate ``.dockerignore`` at the repo root keeps the context small.
 
 FROM python:3.11-slim
 
+ENV PYTHONUNBUFFERED=1
+
 WORKDIR /app
 
 # Dependency layer: copy only what setup.py needs so changes to the demo script do
@@ -15,7 +17,6 @@ COPY ydb ./ydb
 COPY examples/opentelemetry/requirements.txt ./examples/opentelemetry/requirements.txt
 RUN pip install --no-cache-dir -e '.[opentelemetry]' -r examples/opentelemetry/requirements.txt
 
-# Demo script.
+# Demo scripts.
 COPY examples/opentelemetry/otel_example.py ./examples/opentelemetry/otel_example.py
-
-CMD ["python", "examples/opentelemetry/otel_example.py"]
+COPY examples/opentelemetry/load_tank.py ./examples/opentelemetry/load_tank.py