-
Notifications
You must be signed in to change notification settings - Fork 187
feat: add metrics reference page infrastructure (data + shortcode + conversion script) #1083
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # Metrics Data | ||
|
|
||
| This directory contains per-version snapshots of Jaeger's metrics in YAML format. | ||
| The files are generated automatically by the jaeger main repo release workflow | ||
| and follow the same versioned directory convention as `data/cli/`. | ||
|
|
||
| ## Structure | ||
|
|
||
| ``` | ||
| data/metrics/ | ||
| {version}/ # e.g. 2.17 | ||
| metrics.yaml # combined metrics snapshot for this release | ||
| ``` | ||
|
|
||
| ## Format | ||
|
|
||
| Each `metrics.yaml` contains a list of metric entries scraped from the | ||
| Prometheus `/metrics` endpoint of the `jaeger` binary during integration tests: | ||
|
|
||
| ```yaml | ||
| version: "2.17.0" | ||
| metrics: | ||
| - name: otelcol_exporter_sent_spans | ||
| labels: | ||
| - exporter | ||
| - service_name | ||
| - service_version | ||
| - name: otelcol_receiver_accepted_spans | ||
| labels: | ||
| - receiver | ||
| - service_name | ||
| - service_version | ||
| - transport | ||
| ``` | ||
|
|
||
| The file is generated by the `scripts/release/publish-metrics-to-docs.sh` | ||
| script in the jaeger main repository, which downloads the combined metrics | ||
| artifact from the release workflow and converts it to this YAML format. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # This file is a placeholder for the next release metrics. | ||
| # It will be replaced by the jaeger release workflow when a new version is published. | ||
| # See data/metrics/README.md for the format. | ||
| version: "next-release" | ||
| metrics: [] |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,59 @@ | ||||||
| {{/* | ||||||
| Shortcode: metrics | ||||||
|
|
||||||
| Renders a reference table of all Jaeger metrics for a given documentation | ||||||
| version, sourced from data/metrics/{version}/metrics.yaml. | ||||||
|
|
||||||
| Mirrors the pattern used by the CLI flags shortcode. | ||||||
|
|
||||||
| Usage in a content .md file: | ||||||
| {{< metrics >}} | ||||||
|
|
||||||
| The version is inferred from the page's path (e.g. /docs/v2/2.17/... → "2.17"). | ||||||
| Falls back to "next-release" when no version can be determined. | ||||||
| */}} | ||||||
|
|
||||||
| {{- $version := "next-release" -}} | ||||||
| {{- $pathParts := split .Page.RelPermalink "/" -}} | ||||||
| {{- range $i, $p := $pathParts -}} | ||||||
| {{- if and (eq $p "v2") (gt (len $pathParts) (add $i 1)) -}} | ||||||
| {{- $version = index $pathParts (add $i 1) -}} | ||||||
| {{- end -}} | ||||||
| {{- end -}} | ||||||
|
|
||||||
| {{- $data := index site.Data.metrics $version -}} | ||||||
| {{- if not $data -}} | ||||||
|
||||||
| {{- if not $data -}} | |
| {{- if and (not $data) (eq $version "next-release") -}} |
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,98 @@ | ||||||||||
| #!/usr/bin/env python3 | ||||||||||
| # Copyright (c) 2026 The Jaeger Authors. | ||||||||||
| # SPDX-License-Identifier: Apache-2.0 | ||||||||||
| """ | ||||||||||
| convert-metrics-to-yaml.py | ||||||||||
|
|
||||||||||
| Converts a combined Prometheus text-format metrics snapshot (produced by the | ||||||||||
| jaeger integration tests) into the YAML data file consumed by the Hugo | ||||||||||
| documentation site. | ||||||||||
|
|
||||||||||
| Usage: | ||||||||||
| python3 scripts/convert-metrics-to-yaml.py \\ | ||||||||||
| --input combined-metrics.txt \\ | ||||||||||
| --output data/metrics/2.17/metrics.yaml \\ | ||||||||||
| --version 2.17.0 | ||||||||||
|
|
||||||||||
| Input format (Prometheus text exposition format): | ||||||||||
| # HELP otelcol_exporter_sent_spans ... | ||||||||||
| # TYPE otelcol_exporter_sent_spans counter | ||||||||||
| otelcol_exporter_sent_spans{exporter="otlp", ...} 42 | ||||||||||
|
|
||||||||||
| Output format: | ||||||||||
| version: "2.17.0" | ||||||||||
| metrics: | ||||||||||
| - name: otelcol_exporter_sent_spans | ||||||||||
| labels: | ||||||||||
| - exporter | ||||||||||
| - service_name | ||||||||||
| """ | ||||||||||
|
|
||||||||||
| import argparse | ||||||||||
| import re | ||||||||||
| import sys | ||||||||||
| from collections import defaultdict | ||||||||||
|
|
||||||||||
| LABEL_PATTERN = re.compile(r'(\w+)=') | ||||||||||
| METRIC_LINE = re.compile(r'^([a-zA-Z_:][a-zA-Z0-9_:]*)(?:\{([^}]*)\})?\s') | ||||||||||
| EXCLUDED_LABELS = {'service_instance_id', 'otel_scope_version', 'otel_scope_schema_url'} | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def parse_metrics(text: str) -> dict: | ||||||||||
| """Parse Prometheus text format and return {metric_name: set_of_label_names}.""" | ||||||||||
| metrics = defaultdict(set) | ||||||||||
| for line in text.splitlines(): | ||||||||||
| line = line.strip() | ||||||||||
| if not line or line.startswith('#'): | ||||||||||
| continue | ||||||||||
| m = METRIC_LINE.match(line) | ||||||||||
| if not m: | ||||||||||
| continue | ||||||||||
| name = m.group(1) | ||||||||||
| labels_str = m.group(2) or '' | ||||||||||
| labels = set(LABEL_PATTERN.findall(labels_str)) - EXCLUDED_LABELS | ||||||||||
| metrics[name].update(labels) | ||||||||||
|
Comment on lines
+36
to
+54
|
||||||||||
| return metrics | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def metrics_to_yaml(metrics: dict, version: str) -> str: | ||||||||||
| """Render metrics dict as YAML string.""" | ||||||||||
| lines = [f'version: "{version}"', 'metrics:'] | ||||||||||
| for name in sorted(metrics): | ||||||||||
| labels = sorted(metrics[name]) | ||||||||||
| lines.append(f' - name: {name}') | ||||||||||
| if labels: | ||||||||||
| lines.append(' labels:') | ||||||||||
| for label in labels: | ||||||||||
| lines.append(f' - {label}') | ||||||||||
| lines.append('') # trailing newline | ||||||||||
| return '\n'.join(lines) | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def main(): | ||||||||||
| parser = argparse.ArgumentParser(description='Convert Prometheus metrics snapshot to Hugo data YAML') | ||||||||||
| parser.add_argument('--input', required=True, help='Path to combined Prometheus text-format metrics file') | ||||||||||
| parser.add_argument('--output', required=True, help='Path to write the output YAML file') | ||||||||||
| parser.add_argument('--version', required=True, help='Jaeger version string, e.g. 2.17.0') | ||||||||||
| args = parser.parse_args() | ||||||||||
|
|
||||||||||
| with open(args.input) as f: | ||||||||||
| text = f.read() | ||||||||||
|
|
||||||||||
| metrics = parse_metrics(text) | ||||||||||
| if not metrics: | ||||||||||
| print(f'ERROR: no metrics found in {args.input}', file=sys.stderr) | ||||||||||
| sys.exit(1) | ||||||||||
|
|
||||||||||
| yaml_out = metrics_to_yaml(metrics, args.version) | ||||||||||
|
|
||||||||||
| import os | ||||||||||
| os.makedirs(os.path.dirname(args.output), exist_ok=True) | ||||||||||
|
||||||||||
| os.makedirs(os.path.dirname(args.output), exist_ok=True) | |
| output_dir = os.path.dirname(args.output) | |
| if output_dir: | |
| os.makedirs(output_dir, exist_ok=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The version inference logic is looking for a "v2" path segment, but this site’s docs URLs are mounted as /docs/{version}/… (e.g. /docs/2.17/…) and /docs/{major}.dev/… for dev docs. As written, $version will never be set for normal versioned pages, so the shortcode will always default to next-release and show the wrong data. Align this with the existing CLI shortcode logic (derive version from
index (split .Page.RelPermalink "/") 2and map *.dev → next-release).