Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redpanda: Add consumer group lag metrics #7

Open
wants to merge 8 commits into
base: redpanda-3
Choose a base branch
from
Open

Redpanda: Add consumer group lag metrics #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b58c0e4
Redpanda: Fix docker repository and sync conf
BenPope Mar 20, 2025
44cd13b
Redpanda: Refactor consumer group metrics
BenPope Mar 18, 2025
7dea760
Redpanda: Bump to v25.1.1-rc2
BenPope Mar 20, 2025
2eaf6aa
Redpanda: Configure group_lag metrics
BenPope Mar 20, 2025
7b73093
Redpanda: Add consumer group lag metrics
BenPope Mar 18, 2025
0988f26
Redpanda: Release 2.1.0
BenPope Mar 18, 2025
7eb3aab
Redpanda: Release 3.0.0
BenPope Mar 20, 2025
6b6f268
Redpanda: Add demo dashboard for consumer_lag
BenPope Mar 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions redpanda/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# CHANGELOG - redpanda

## 3.0.0 / 2025-03-20

***Removed***:

* Removed kafka.group_offset from default metrics.

## 2.1.0 / 2025-03-20

***Added***:

* Added metrics for consumer group lag.

## 2.0.0 / 2023-08-21

***Added***:
Expand Down
2 changes: 1 addition & 1 deletion redpanda/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ To build an updated version of the Agent:
```dockerfile
FROM gcr.io/datadoghq/agent:latest

ARG INTEGRATION_VERSION=2.0.0
ARG INTEGRATION_VERSION=3.0.0

RUN agent integration install -r -t datadog-redpanda==${INTEGRATION_VERSION}
```
Expand Down
1 change: 1 addition & 0 deletions redpanda/assets/configuration/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ files:
- redpanda.pandaproxy
- redpanda.scheduler
- redpanda.schemaregistry
- redpanda.kafka.consumer_group_offset
- template: instances/openmetrics
overrides:
openmetrics_endpoint.value.example: http://localhost:9644/public_metrics
Expand Down
226 changes: 226 additions & 0 deletions redpanda/assets/dashboards/consumer_lag.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
{
"title": "consumer_lag",
"description": "[[suggested_dashboards]]",
"widgets": [
{
"id": 5613612194856358,
"definition": {
"title": "",
"title_size": "16",
"title_align": "left",
"show_legend": true,
"legend_layout": "auto",
"legend_columns": [
"avg",
"min",
"max",
"value",
"sum"
],
"time": {},
"type": "timeseries",
"requests": [
{
"formulas": [
{
"alias": "a",
"formula": "exclude_null(query1)"
}
],
"queries": [
{
"name": "query1",
"data_source": "metrics",
"query": "sum:redpanda.kafka.partition_committed_offset{*} by {redpanda_topic}"
}
],
"response_format": "timeseries",
"style": {
"palette": "dog_classic",
"order_by": "values",
"line_type": "solid",
"line_width": "normal"
},
"display_type": "line"
}
],
"yaxis": {
"include_zero": true,
"min": "-1"
}
},
"layout": {
"x": 0,
"y": 0,
"width": 4,
"height": 2
}
},
{
"id": 4806429120428468,
"definition": {
"title": "",
"title_size": "16",
"title_align": "left",
"show_legend": true,
"legend_layout": "auto",
"legend_columns": [
"avg",
"min",
"max",
"value",
"sum"
],
"time": {},
"type": "timeseries",
"requests": [
{
"formulas": [
{
"alias": "a",
"formula": "exclude_null(query1)"
}
],
"queries": [
{
"name": "query1",
"data_source": "metrics",
"query": "sum:redpanda.kafka.group_lag_max{*} by {redpanda_group}"
}
],
"response_format": "timeseries",
"style": {
"palette": "dog_classic",
"order_by": "values",
"line_type": "solid",
"line_width": "normal"
},
"display_type": "line"
}
],
"yaxis": {
"include_zero": true,
"min": "-1"
}
},
"layout": {
"x": 4,
"y": 0,
"width": 4,
"height": 2
}
},
{
"id": 967937420665096,
"definition": {
"title": "",
"title_size": "16",
"title_align": "left",
"show_legend": true,
"legend_layout": "auto",
"legend_columns": [
"avg",
"min",
"max",
"value",
"sum"
],
"time": {},
"type": "timeseries",
"requests": [
{
"formulas": [
{
"alias": "a",
"formula": "exclude_null(query1)"
}
],
"queries": [
{
"name": "query1",
"data_source": "metrics",
"query": "sum:redpanda.kafka.group_offset{*} by {redpanda_group,redpanda_topic,redpanda_partition}"
}
],
"response_format": "timeseries",
"style": {
"palette": "dog_classic",
"order_by": "values",
"line_type": "solid",
"line_width": "normal"
},
"display_type": "line"
}
],
"yaxis": {
"include_zero": true,
"min": "-1"
}
},
"layout": {
"x": 0,
"y": 2,
"width": 4,
"height": 2
}
},
{
"id": 1669608188096820,
"definition": {
"title": "",
"title_size": "16",
"title_align": "left",
"show_legend": true,
"legend_layout": "auto",
"legend_columns": [
"avg",
"min",
"max",
"value",
"sum"
],
"time": {},
"type": "timeseries",
"requests": [
{
"formulas": [
{
"alias": "a",
"formula": "exclude_null(query1)"
}
],
"queries": [
{
"name": "query1",
"data_source": "metrics",
"query": "sum:redpanda.kafka.group_lag_sum{*} by {redpanda_group}"
}
],
"response_format": "timeseries",
"style": {
"palette": "dog_classic",
"order_by": "values",
"line_type": "solid",
"line_width": "normal"
},
"display_type": "line"
}
],
"yaxis": {
"include_zero": true,
"min": "-1"
}
},
"layout": {
"x": 4,
"y": 2,
"width": 4,
"height": 2
}
}
],
"template_variables": [],
"layout_type": "ordered",
"notify_list": [],
"reflow_type": "fixed"
}
2 changes: 1 addition & 1 deletion redpanda/datadog_checks/redpanda/__about__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '2.0.0'
__version__ = '3.0.0'
11 changes: 11 additions & 0 deletions redpanda/datadog_checks/redpanda/data/conf.yaml.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ instances:
# - redpanda.pandaproxy
# - redpanda.scheduler
# - redpanda.schemaregistry
# - redpanda.kafka.consumer_group_offset

## @param raw_metric_prefix - string - optional
## A prefix that is removed from all exposed metric names, if present.
Expand Down Expand Up @@ -541,6 +542,16 @@ instances:
# - TLSv1.2
# - TLSv1.3

## @param tls_ciphers - list of strings - optional
## The list of ciphers suites to use when connecting to an endpoint. If not specified,
## `ALL` ciphers are used. For list of ciphers see:
## https://www.openssl.org/docs/man1.0.2/man1/ciphers.html
#
# tls_ciphers:
# - TLS_AES_256_GCM_SHA384
# - TLS_CHACHA20_POLY1305_SHA256
# - TLS_AES_128_GCM_SHA256

## @param headers - mapping - optional
## The headers parameter allows you to send specific headers with every request.
## You can use it for explicitly specifying the host header or adding headers for
Expand Down
22 changes: 21 additions & 1 deletion redpanda/datadog_checks/redpanda/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,28 @@
'redpanda_kafka_request_latency_seconds': 'kafka.request_latency_seconds',
'redpanda_kafka_request_bytes': 'kafka.request_bytes',
'redpanda_kafka_under_replicated_replicas': 'kafka.under_replicated_replicas',
'redpanda_kafka_consumer_group_committed_offset': 'kafka.group_offset',
}

# As of Redpanda v25.1, can be disabled by removing "group"
# from cluster config "enable_consumer_group_metrics"
REDPANDA_KAFKA_CONSUMER_GROUP_INFO = {
'redpanda_kafka_consumer_group_consumers': 'kafka.group_count',
'redpanda_kafka_consumer_group_topics': 'kafka.group_topic_count',
}

# As of Redpanda v25.1, can be disabled by removing "partition"
# from cluster config "enable_consumer_group_metrics"
REDPANDA_KAFKA_CONSUMER_GROUP_OFFSET = {
'redpanda_kafka_consumer_group_committed_offset': 'kafka.group_offset',
}

# As of Redpanda v25.1, can be enabled by adding "consumer_lag"
# to cluster config "enable_consumer_group_metrics"
REDPANDA_KAFKA_CONSUMER_GROUP_LAG = {
'redpanda_kafka_consumer_group_lag_sum': 'kafka.group_lag_sum',
'redpanda_kafka_consumer_group_lag_max': 'kafka.group_lag_max',
}

REDPANDA_MEMORY = {
'redpanda_memory_allocated_memory': 'memory.allocated_memory',
'redpanda_memory_available_memory': 'memory.available_memory',
Expand Down Expand Up @@ -128,6 +145,8 @@
REDPANDA_CLUSTER_PARTITION,
REDPANDA_IO_QUEUE,
REDPANDA_KAFKA,
REDPANDA_KAFKA_CONSUMER_GROUP_INFO,
REDPANDA_KAFKA_CONSUMER_GROUP_LAG,
REDPANDA_MEMORY,
REDPANDA_RAFT,
REDPANDA_REACTOR,
Expand All @@ -142,4 +161,5 @@
'redpanda.pandaproxy': REDPANDA_PANDAPROXY,
'redpanda.scheduler': REDPANDA_SCHEDULER,
'redpanda.schemaregistry': REDPANDA_SCHEMA_REGISTRY,
'redpanda.kafka.consumer_group_offset': REDPANDA_KAFKA_CONSUMER_GROUP_OFFSET,
}
4 changes: 2 additions & 2 deletions redpanda/hatch.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

[[envs.default.matrix]]
python = ["3.12"]
version = ["23.1"]
version = ["25.1"]

[envs.default.overrides]
matrix.version.env-vars = [
{ key = "REDPANDA_VERSION", value = "v23.1.10", if = ["23.1"] },
{ key = "REDPANDA_VERSION", value = "v25.1.1-rc2", if = ["25.1"] },
]

[envs.default.env-vars]
Expand Down
2 changes: 2 additions & 0 deletions redpanda/metadata.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ redpanda.io_queue.total_write_ops,count,,operation,,Total write operations passe
redpanda.kafka.group_offset,gauge,,,,Consumer group committed offset,-1,redpanda,kafka group offset ,
redpanda.kafka.group_count,gauge,,,,Number of consumers in a group,-1,redpanda,kafka group count ,
redpanda.kafka.group_topic_count,gauge,,,,Number of topics in a group,-1,redpanda,kafka group topic count ,
redpanda.kafka.group_lag_max,gauge,,,,Maximum lag for any partition in the group,-1,redpanda,kafka group lag max ,
redpanda.kafka.group_lag_sum,gauge,,,,Sum of lag for all partitions in the group,-1,redpanda,kafka group lag sum ,
redpanda.cluster.partitions,gauge,,,,Configured number of partitions for the topic,-1,redpanda,cluster partitions ,
redpanda.cluster.replicas,gauge,,,,Configured number of replicas for the topic,-1,redpanda,cluster replicas ,
redpanda.kafka.request_latency_seconds,gauge,,second,,Internal latency of kafka produce requests,-1,redpanda,kafka request latency seconds ,
Expand Down
Loading
Loading