Merge remote-tracking branch 'origin/main' into krajo/om2.0-native-histograms

Author: krajorama

blog/posts/2025-12-02-visualizing-target-relabeling-rules-in-the-prometheus-ui.md

@@ -0,0 +1,39 @@
+---
+title: Visualizing Target Relabeling Rules in Prometheus 3.8.0
+created_at: 2025-12-02
+kind: article
+author_name: Julius Volz (@juliusv)
+---
+
+Prometheus' [target relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) feature allows you to adjust the labels of a discovered target or even drop the target entirely. Relabeling rules, while powerful, can be hard to understand and debug. Your rules have to match the expected labels that your service discovery mechanism returns, and getting any step wrong could label your target incorrectly or accidentally drop it.
+
+To help you figure out where things go wrong (or right), Prometheus 3.8.0 [just added a relabeling visualizer](https://github.com/prometheus/prometheus/pull/17337) to the Prometheus server's web UI that allows you to inspect how each relabeling rule is applied to a discovered target's labels. Let's take a look at how it works!
+
+<!-- more -->
+
+## Using the relabeling visualizer
+
+If you head to any Prometheus server's "Service discovery" page (for example: https://demo.promlabs.com/service-discovery), you will now see a new "show relabeling" button for each discovered target:
+
+![Service discovery page screenshot](/assets/blog/2025-12-02/prometheus-sd-page-show-relabeling.png)
+
+Clicking this button shows you how each relabeling rule is applied to that particular target in sequence:
+
+<video playsinline muted autoplay loop controls>
+  <source src="/assets/blog/2025-12-02/prometheus-sd-page-relabeling-visualizer.mp4" type="video/mp4" />
+  <p>Sorry, your browser does not support videos.</p>
+</video>
+
+The visualizer shows you:
+
+* The **initial labels** of the target as discovered by the service discovery mechanism.
+* The details of **each relabeling rule**, including its action type and other parameters.
+* **How the labels change** after each relabeling rule is applied, with changes, additions, and deletions highlighted in color.
+* Whether the target is ultimately **kept or dropped** after all relabeling rules have been applied.
+* The **final output labels** of the target if it is kept.
+
+To debug your relabeling rules, you can now read this diagram from top to bottom and find the exact step where the labels change in an unexpected way or where the target gets dropped. This should help you identify misconfigurations in your relabeling rules more easily.
+
+## Conclusion
+
+The new relabeling visualizer in the Prometheus server's web UI is a powerful tool to help you understand and debug your target relabeling configurations. By providing a step-by-step view of how each relabeling rule affects a target's labels, it makes it easier to identify and fix issues in your setup. Update your Prometheus servers to 3.8.0 now to give it a try!

docs/instrumenting/exposition_formats.md

@@ -167,6 +167,18 @@ When such features are enabled either by feature flag
 appropriate configuration option (`scrape_native_histograms: true`) then
 Protobuf will be favored over other exposition formats.
 
+## HTTP Content-Type requirements
+
+Starting with Prometheus 3.0, scrape targets **must** return a valid `Content-Type` header for the metrics endpoint. If the `Content-Type` is missing, unparsable, or not a supported media type, **the scrape will fail**. See changes in [scrape protocols](https://prometheus.io/docs/prometheus/latest/migration/#scrape-protocols) in the migration guide for details.
+
+See each of the exposition format sections for the accurate HTTP content types.
+
+### ScrapeProtocols vs Content-Type
+
+Prometheus scrape config offers scrape protocol negotiation based on the content-type using the [`scrape_protocols`](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) config. For the Prometheus user convenience the scrape protocols are referenced by a unique name that maps to the concrete content-type. See [Protocol Headers](./content_negotiation.md#protocol-headers) for details.
+
+However, the targets should expose metrics in the exposition format with the absolute, response content-type (e.g. `application/openmetrics-text;version=1.0.0`) and only one.
+
 ## Historical versions
 
 For details on historical format versions, see the legacy

docs/specs/om/open_metrics_spec_2_0.md

@@ -571,6 +571,39 @@ acme_http_request_seconds_sum{path="/api/v1",method="GET"} 1.2e2 [email protected]
 acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="0.5"} 1 [email protected]
 acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="1"} 2 [email protected]
 acme_http_request_seconds_buckets{path="/api/v1",method="GET",le="+Inf"} 2 [email protected]
+# TYPE "foodb.read.errors" counter
+# HELP "foodb.read.errors" The number of errors in the read path for fooDb.
+{"foodb.read.errors","service.name"="my_service"} 3482
+# EOF
+```
+
+##### UTF-8 Quoting
+
+Metric names not conforming to the ABNF definition of `metricname` MUST be
+enclosed in double quotes and the alternative UTF-8 syntax MUST be used. In
+these MetricPoints, the quoted metric name MUST be moved inside the brackets
+without a label name and equal sign, in accordance with the ABNF. The metric
+names MUST be enclosed in double quotes in TYPE, UNIT, and HELP lines. Quoting
+and the alternative metric syntax MAY be used for any metric name, regardless of
+whether the name requires quoting or not.
+
+Label names not conforming to the `label-name` ABNF definition MUST be enclosed
+in double quotes. Any label name MAY be enclosed in double quotes.
+
+Expressed as regular expressions, metric names that don't need to be enclosed
+in quotes must match: `^[a-zA-Z_:][a-zA-Z0-9_:]*$`. For label names, the string
+must match: `^[a-zA-Z_][a-zA-Z0-9_]*$`.
+
+Complete example:
+
+```openmetrics
+# TYPE "process.cpu.seconds" counter
+# UNIT "process.cpu.seconds" seconds
+# HELP "process.cpu.seconds" Total user and system CPU time spent in seconds.
+{"process.cpu.seconds","node.name"="my_node"} 4.20072246e+06
+# TYPE "quoting_example" gauge
+# HELP "quoting_example" Number of goroutines that currently exist.
+{"quoting_example","foo"="bar"} 4.5
 # EOF
 ```
 
@@ -674,6 +707,8 @@ The value of a UNIT or HELP line MAY be empty. This MUST be treated as if no met
 # HELP foo_seconds Some text and \n some \" escaping
 ```
 
+See the UTF-8 Quoting section for circumstances where the metric name MUST be enclosed in double quotes.
+
 There MUST NOT be more than one of each type of metadata line for a MetricFamily. The ordering SHOULD be TYPE, UNIT, HELP.
 
 Aside from this metadata and the EOF line at the end of the message, you MUST NOT expose lines beginning with a #.
@@ -702,6 +737,13 @@ Label values MAY be any valid UTF-8 value, so escaping MUST be applied as per th
 bar_seconds_count{a="x",b="escaping\" example \n "} 0
 ```
 
+Metric names and label names MAY also be any valid UTF-8 value, and under certain circumstances they MUST be quoted and escaped per the ABNF.
+See the UTF-8 Quoting section for specifics.
+
+```openmetrics-add-eof
+{"\"bar\".seconds.count","b\\"="escaping\" example \n "} 0
+```
+
 The rendering of values for a MetricPoint can include additional labels (e.g. the "le" label for a Histogram type), which MUST be rendered in the same way as a Metric's own LabelSet.
 
 #### MetricPoint

public/assets/blog/2025-12-02/prometheus-sd-page-relabeling-visualizer.mp4

@@ -11,6 +11,7 @@ import {
 import TOC from "@/components/TOC";
 import LeftNav from "./LeftNav";
 import { IconMenu2 } from "@tabler/icons-react";
+import { AnchorScroller } from "@/components/AnchorScroller";
 
 export default function DocsLayout({
   children,
@@ -85,6 +86,7 @@ export default function DocsLayout({
           }}
         />
       </Group>
+      <AnchorScroller />
     </>
   );
 }

public/assets/blog/2025-12-02/prometheus-sd-page-show-relabeling.png

@@ -53,9 +53,10 @@ pre > code > span {
 }
 
 .markdown-content {
-  img:not(.no-markdown-image-styles) {
+  img:not(.no-markdown-image-styles),
+  video {
     max-width: 90%;
-    max-height: 50vh;
+    max-height: 65vh;
     height: auto;
     margin: 1.5em auto;
     display: block;

src/app/docs/layout.tsx

@@ -0,0 +1,31 @@
+"use client";
+
+import { useEffect } from "react";
+
+// A component that, when included on a page, scrolls to the element
+// indicated by the URL hash (if any) after the page loads. While the
+// browser normally does this automatically, it does so too early in
+// our case, before the layout settles, resulting in incorrect scroll
+// positions.
+export function AnchorScroller() {
+  useEffect(() => {
+    const hash = window.location.hash;
+    if (!hash) {
+      return;
+    }
+
+    const el = document.querySelector(hash);
+    if (!el) {
+      return;
+    }
+
+    // Wait for layout to settle using two wrapped requestAnimationFrame calls.
+    requestAnimationFrame(() => {
+      requestAnimationFrame(() => {
+        el.scrollIntoView({ block: "start" });
+      });
+    });
+  }, []);
+
+  return null;
+}

src/app/globals.css

@@ -147,10 +147,10 @@ export default async function PromMarkdown({
                   // <Group> with display: "inline-flex" somehow breaks link underlining,
                   // so going for this manual solution instead.
                   <span>
-                    {children}
+                    {children}&nbsp;
                     <IconExternalLink
                       size="0.9em"
-                      style={{ marginLeft: 4, marginBottom: -1.5 }}
+                      style={{ marginBottom: -1.5 }}
                     />
                   </span>
                 ) : (