Update assemblies and modules Docs for Metrics Reporter integration #11329

OwenCorrigan76 · 2025-04-07T15:54:53Z

Type of change

Documentation

Description

With newly added support for the Strimzi Metrics Reporter to Kafka brokers/controllers components, this PR updates the assemblies and modules documents to reflect the changes in the way we can deploy metrics to the Kafka brokers and controllers.

Updates:

Example metrics files
Example grafana dashboards – mentioning metrics reporter-specific grafana dashboards

Structure for new content:

Assembly file: Using Metrics Reporter with Strimzi
Procedure file: Enabling Metrics Reporter
Reference file: Example Metrics Reporter configuration

Related to #11051

Checklist

Please go through this checklist and make sure all applicable tasks have been done

Write tests
Make sure all tests pass
Update documentation
Check RBAC rights for Kubernetes / OpenShift roles
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md
Supply screenshots for visual changes, such as Grafana dashboards

fvaleri · 2025-04-08T08:01:02Z

documentation/assemblies/metrics/assembly-metrics-config-files.adoc

-<13> Metrics configuration that defines Prometheus JMX Exporter relabeling rules for Kafka.
-<14> Metrics configuration that defines Prometheus JMX Exporter relabeling rules for MirrorMaker 2.
-<15> Metrics configuration that defines Prometheus JMX Exporter relabeling rules for OAuth 2.0.
+<10> Example Grafana dashboards with Metrics Reporter for the Kafka component.


Suggested change

<10> Example Grafana dashboards with Metrics Reporter for the Kafka component.

<10> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.

fvaleri · 2025-04-08T08:02:24Z

documentation/assemblies/metrics/assembly-metrics-config-files.adoc

-<14> Metrics configuration that defines Prometheus JMX Exporter relabeling rules for MirrorMaker 2.
-<15> Metrics configuration that defines Prometheus JMX Exporter relabeling rules for OAuth 2.0.
+<10> Example Grafana dashboards with Metrics Reporter for the Kafka component.
+<11> Resource for deploying and configuring Strimzi Metrics Reporter based Kafka.


Suggested change

<11> Resource for deploying and configuring Strimzi Metrics Reporter based Kafka.

<11> Resource for deploying and configuring Kafka with Strimzi Metrics Reporter.

fvaleri · 2025-04-08T08:06:20Z

documentation/modules/metrics/ref-prometheus-metrics-config.adoc

+Grafana dashboards are dependent on relabeling rules, which are defined for Strimzi components in the custom resource configuration.

 A label is a name-value pair.
 Relabeling is the process of writing a label dynamically.
 For example, the value of a label may be derived from the name of a Kafka server and client ID.

 Strimzi provides example custom resource configuration YAML files with relabeling rules.
-When deploying Prometheus metrics configuration, you can can deploy the example custom resource or copy the metrics configuration to your own custom resource definition.


Only Prometheus JMX Exporter has relabeling rules, so I think we should make it clear. What about moving this part to the paragraph titled "Example Prometheus Jmx Exporter metrics configuration"?

fvaleri · 2025-04-08T08:09:46Z

documentation/assemblies/metrics/assembly_metrics-prometheus-setup.adoc

-Using Prometheus with Strimzi, requires the following:
+You can enable either Prometheus JMX Exporter or Strimzi Metrics Reporter to expose metrics:
+
+xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Enabling Prometheus Jmx metrics through configuration]


Suggested change

xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Enabling Prometheus Jmx metrics through configuration]

xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Enabling Prometheus JMX Exporter through configuration]

fvaleri · 2025-04-08T08:10:25Z

documentation/modules/metrics/proc-jmx-exporter-metrics-kafka-deploy-options.adoc

-[id='proc-metrics-kafka-deploy-options-{context}']
-= Enabling Prometheus metrics through configuration
+[id='proc-jmx-exporter-metrics-kafka-deploy-options-{context}']
+= Enabling Prometheus Jmx metrics through configuration


Suggested change

= Enabling Prometheus Jmx metrics through configuration

= Enabling Prometheus JMX Exporter through configuration

fvaleri · 2025-04-08T08:12:27Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+// metrics/assembly_metrics-kafka.adoc
+
+[id='proc-metrics-reporter-kafka-deploy-options-{context}']
+= Enabling Metrics Reporter metrics through configuration


Suggested change

= Enabling Metrics Reporter metrics through configuration

= Enabling Strimzi Metrics Reporter through configuration

fvaleri · 2025-04-08T08:15:51Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+To enable and expose metrics in Strimzi for Prometheus, use metrics configuration properties.
+
+The following components require `metricsConfig` configuration to expose metrics:


Suggested change

To enable and expose metrics in Strimzi for Prometheus, use metrics configuration properties.

The following components require `metricsConfig` configuration to expose metrics:

The following components require `metricsConfig` configuration to expose metrics in Prometheus format:

fvaleri · 2025-04-08T08:16:26Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+This configuration enables the {StrimziMetricsReporter} to expose metrics through an HTTP endpoint.
+The port for the Strimzi Metrics Reporter HTTP endpoint is 9404.


Suggested change

This configuration enables the {StrimziMetricsReporter} to expose metrics through an HTTP endpoint.

The port for the Strimzi Metrics Reporter HTTP endpoint is 9404.

This configuration enables the {StrimziMetricsReporter} to expose metrics through an HTTP endpoint on port 9404.

and similar change for the Prometheus JMX Exporter paragraph.

fvaleri · 2025-04-08T08:18:13Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+spec:
+  kafka:
+    # ...
+    metricsConfig: <1>


There is no reference for <1>. Is it really needed?

fvaleri · 2025-04-08T08:19:03Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+
+  # _metrics configuration..._


Do we need this?

ppatierno · 2025-04-08T09:52:27Z

documentation/assemblies/metrics/assembly-metrics-config-files.adoc

+├── kafka-cruise-control-metrics.yaml <14>
+├── kafka-metrics.yaml <15>
+├── kafka-mirror-maker-2-metrics.yaml <16>
+└── oauth-metrics.yaml <17>


a couple of questions here:

aren't we getting examples for all the other components (apart Kafka) for using the Strimzi Metrics reporter?

wouldn't be better to put the Prometheus JMX exporter related example in a dedicated folder now that we have the strimzi-metrics-reporter one?

Yeah that was the idea thanks @OwenCorrigan76
@fvaleri @PaulRMellor wdyt?

LGTM, but the folder name should be prometheus-jmx-exporter and we need a similar README.md for consistency.

Yes. I'll update this docs PR and then update the main integration PR. Thanks

PaulRMellor

Looking good, Owen.
I left some suggestions.
There's a bit of overlap with a PR I'm about to merge in, which I've pointed out.
I also left a note on Kafka Exporter -- I've made a note to do something about that when we get these updates merged in

PaulRMellor · 2025-04-10T15:22:05Z

documentation/assemblies/metrics/assembly-metrics-config-files.adoc

+<8> PodMonitor definitions translated by the Prometheus Operator into jobs for the Prometheus server to be able to scrape metrics data directly from pods.
+<9> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.
+<10> Resource for deploying and configuring Kafka with Strimzi Metrics Reporter.
+<11> Example Grafana dashboards for the different Strimzi components.


Suggested change

<11> Example Grafana dashboards for the different Strimzi components.

<11> Grafana dashboards for components using the JMX Reporter.

PaulRMellor · 2025-04-10T15:32:05Z

documentation/assemblies/metrics/assembly-metrics.adoc

 * xref:assembly-metrics-setup-{context}[Set up and deploy Prometheus to expose metrics]
-* xref:proc-metrics-kafka-deploy-options-{context}[Deploy Kafka Exporter to provide additional metrics]
+* xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Deploy Kafka Exporter to provide additional metrics]
 * xref:proc-metrics-grafana-dashboard-{context}[Use Grafana to present the Prometheus metrics]


I deleted this section in PR #11307
I don't think we need it here, so would revert the changes to this file

PaulRMellor · 2025-04-10T15:43:37Z

documentation/assemblies/metrics/assembly-metrics-config-files.adoc

+<6> Alerting rules examples for use with Prometheus Alertmanager (deployed with Prometheus).
+<7> Installation resource file for the Prometheus image.
+<8> PodMonitor definitions translated by the Prometheus Operator into jobs for the Prometheus server to be able to scrape metrics data directly from pods.
+<9> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.


Suggested change

<9> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.

<9> Grafana dashboards for components using the Strimzi Metrics Reporter.

PaulRMellor · 2025-04-10T15:53:39Z

documentation/assemblies/metrics/assembly_metrics-prometheus-setup.adoc

+
+xref:proc-metrics-reporter-kafka-deploy-options-{context}[Enabling Strimzi Metrics Reporter through configuration]
+
+Once the type is determined, using Prometheus with Strimzi, requires the following:


Suggested change

Once the type is determined, using Prometheus with Strimzi, requires the following:

After enabling metrics, you can integrate with Prometheus:

PaulRMellor · 2025-04-10T15:56:19Z

documentation/assemblies/metrics/assembly_metrics-prometheus-setup.adoc

@@ -12,16 +12,23 @@ To expose metrics in Prometheus format, you add configuration to a custom resour
 You must also make sure that the metrics are scraped by your monitoring stack.
 Prometheus and Prometheus Alertmanager are used in the examples provided by Strimzi, but you can use also other compatible tools.

-Using Prometheus with Strimzi, requires the following:
+You can enable either Prometheus JMX Exporter or Strimzi Metrics Reporter to expose metrics:


Suggested change

You can enable either Prometheus JMX Exporter or Strimzi Metrics Reporter to expose metrics:

You can expose metrics using one of the following approaches:

PaulRMellor · 2025-04-10T16:25:25Z

documentation/modules/metrics/proc-strimzi-metrics-reporter-kafka-deploy-options.adoc

+It is a good starting point for trying Prometheus with Strimzi.
+
+This procedure shows how to deploy example Prometheus metrics configuration in the `Kafka` resource.
+The process is the same when deploying the example files for other resources.


Suggested change

The process is the same when deploying the example files for other resources.

we include a note/cross-ref on how to configure the Kafka Exporter as well in the JMX procedure. Do we want it here too?

Lines 28-31: https://github.com/strimzi/strimzi-kafka-operator/pull/11307/files#diff-80eda6a9cce800533455d246c9b923ba06d53ea83645996177b63b5ef4f33e66R28-R31

Or maybe that needs to be separated from both now, so we link from each of these procedures. Note taken

PaulRMellor · 2025-04-10T16:33:09Z

documentation/modules/metrics/proc_metrics-grafana-dashboard.adoc

@@ -29,7 +29,7 @@ When accessing a dashboard, you can use the `port-forward` command to forward tr
 The name of the Grafana pod is different for each user.

 .Prerequisites
-* xref:proc-metrics-kafka-deploy-options-{context}[Metrics are configured for the Kafka cluster resource]
+* xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Metrics are configured for the Kafka cluster resource]


should we have links for both options here as an either/or?

PaulRMellor · 2025-04-10T16:33:55Z

documentation/modules/metrics/ref-prometheus-jmx-metrics-config.adoc

+
+[id='ref-jmx-metrics-prometheus-metrics-config-{context}']
+
+= Example Prometheus Jmx Exporter metrics configuration


Suggested change

= Example Prometheus Jmx Exporter metrics configuration

= Example Prometheus JMX Exporter metrics configuration

PaulRMellor · 2025-04-10T16:35:47Z

documentation/modules/metrics/ref-prometheus-metrics-config.adoc

-A label is a name-value pair.
-Relabeling is the process of writing a label dynamically.
-For example, the value of a label may be derived from the name of a Kafka server and client ID.
+Strimzi uses the {StrimziMetricsReporter} or the {JMXExporter} to expose metrics through an HTTP endpoint, which can be scraped by the Prometheus server.


I wonder if this file is worth having for two sentences?
Maybe we can just add to the intro of the example metrics file section?

PaulRMellor · 2025-04-10T16:38:57Z

documentation/shared/attributes.adoc

@@ -42,6 +42,7 @@
 //Monitoring links
 :GrafanaHome: link:https://grafana.com/[Grafana Labs^]
 :JMXExporter: link:https://github.com/prometheus/jmx_exporter[Prometheus JMX Exporter^]
+:StrimziMetricsReporter: link:https://github.com/strimzi/metrics-reporter[Strimzi Metrics Reporter^]


do we want to link to this? We only use the link in one place. Maybe we can live without it?

Signed-off-by: OwenCorrigan76 <[email protected]>

fvaleri reviewed Apr 8, 2025

View reviewed changes

fvaleri requested review from PaulRMellor and a team April 8, 2025 08:20

fvaleri added this to the 0.46.0 milestone Apr 8, 2025

fvaleri added the needs review label Apr 8, 2025

ppatierno reviewed Apr 8, 2025

View reviewed changes

OwenCorrigan76 force-pushed the Docs_for_integrate_metrics_reporter branch from e5596ab to 9fe4d31 Compare April 10, 2025 12:42

OwenCorrigan76 requested review from fvaleri and ppatierno April 10, 2025 16:04

PaulRMellor reviewed Apr 10, 2025

View reviewed changes

OwenCorrigan76 added 4 commits April 11, 2025 12:05

Update assemblies and modules docs for Metrics Reporter integration

81a47f6

Signed-off-by: OwenCorrigan76 <[email protected]>

Address Fede's comments

85be22c

Signed-off-by: OwenCorrigan76 <[email protected]>

Create prometheus-jmx-exporter directory in metrcs example

7ac7a23

Signed-off-by: OwenCorrigan76 <[email protected]>

Address Paul's comments

8fa4e37

Signed-off-by: OwenCorrigan76 <[email protected]>

OwenCorrigan76 force-pushed the Docs_for_integrate_metrics_reporter branch from 99e88b1 to 8fa4e37 Compare April 11, 2025 12:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update assemblies and modules Docs for Metrics Reporter integration #11329

Update assemblies and modules Docs for Metrics Reporter integration #11329

OwenCorrigan76 commented Apr 7, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

fvaleri Apr 8, 2025

ppatierno Apr 8, 2025

OwenCorrigan76 Apr 10, 2025

ppatierno Apr 10, 2025

fvaleri Apr 10, 2025

OwenCorrigan76 Apr 10, 2025

PaulRMellor left a comment

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

PaulRMellor Apr 10, 2025

	<10> Example Grafana dashboards with Metrics Reporter for the Kafka component.
	<10> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.

	<11> Resource for deploying and configuring Strimzi Metrics Reporter based Kafka.
	<11> Resource for deploying and configuring Kafka with Strimzi Metrics Reporter.

	xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Enabling Prometheus Jmx metrics through configuration]
	xref:proc-jmx-exporter-metrics-kafka-deploy-options-{context}[Enabling Prometheus JMX Exporter through configuration]

	= Enabling Prometheus Jmx metrics through configuration
	= Enabling Prometheus JMX Exporter through configuration

	= Enabling Metrics Reporter metrics through configuration
	= Enabling Strimzi Metrics Reporter through configuration

		To enable and expose metrics in Strimzi for Prometheus, use metrics configuration properties.

		The following components require `metricsConfig` configuration to expose metrics:

		This configuration enables the {StrimziMetricsReporter} to expose metrics through an HTTP endpoint.
		The port for the Strimzi Metrics Reporter HTTP endpoint is 9404.

	<11> Example Grafana dashboards for the different Strimzi components.
	<11> Grafana dashboards for components using the JMX Reporter.

	<9> Example Grafana dashboards to be used with Metrics Reporter for the Kafka component.
	<9> Grafana dashboards for components using the Strimzi Metrics Reporter.


		xref:proc-metrics-reporter-kafka-deploy-options-{context}[Enabling Strimzi Metrics Reporter through configuration]

		Once the type is determined, using Prometheus with Strimzi, requires the following:

	Once the type is determined, using Prometheus with Strimzi, requires the following:
	After enabling metrics, you can integrate with Prometheus:

	You can enable either Prometheus JMX Exporter or Strimzi Metrics Reporter to expose metrics:
	You can expose metrics using one of the following approaches:


		[id='ref-jmx-metrics-prometheus-metrics-config-{context}']

		= Example Prometheus Jmx Exporter metrics configuration

Update assemblies and modules Docs for Metrics Reporter integration #11329

Are you sure you want to change the base?

Update assemblies and modules Docs for Metrics Reporter integration #11329

Conversation

OwenCorrigan76 commented Apr 7, 2025

Type of change

Description

Updates:

Structure for new content:

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PaulRMellor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment