Skip to content

[improve][doc] Add broker level metrics statistics and expose to prometheus #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yyj8 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[improve][doc] Add broker level metrics statistics and expose to prometheus #1

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e6ce960
[improve][doc] Add broker level metrics statistics and expose to prom…
yyj8 Mar 18, 2023
54dff35
[improve][doc] Add broker level metrics statistics and expose to prom…
yyj8 Mar 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions docs/reference-metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,19 @@ All the broker metrics are labeled with the following labels:
| pulsar_connection_closed_total_count | Gauge | The total number of closed connections. |
| pulsar_broker_throttled_connections | Gauge | The number of throttled connections. |
| pulsar_broker_throttled_connections_global_limit | Gauge | The number of throttled connections due to per-connection limit. |
| pulsar_broker_topics_count | Gauge | The number of Pulsar topics in this broker. |
| pulsar_broker_subscriptions_count | Gauge | The number of Pulsar subscriptions in this broker. |
| pulsar_broker_producers_count | Gauge | The number of active producers connected to this broker. |
| pulsar_broker_consumers_count | Gauge | The number of active consumers connected to this broker. |
| pulsar_broker_rate_in | Gauge | The total message rate coming into this broker (message per second). |
| pulsar_broker_rate_out | Gauge | The total message rate going out from this broker (message per second). |
| pulsar_broker_throughput_in | Gauge | The total throughput coming into this broker (byte per second). |
| pulsar_broker_throughput_out | Gauge | The total throughput going out from this broker (byte per second). |
| pulsar_broker_storage_size | Gauge | The total storage size of all topics in this broker (in bytes). |
| pulsar_broker_storage_logical_size | Gauge | The storage size of all topics in this broker without replicas (in bytes). |
| pulsar_broker_storage_write_rate | Gauge | The total message batches (entries) written to the storage for this broker (message batch per second). |
| pulsar_broker_storage_read_rate | Gauge | The total message batches (entries) read from the storage for this broker (message batch per second). |
| pulsar_broker_msg_backlog | Gauge | The total number of message backlogs in this broker (entries). |

### BookKeeper client metrics

Expand Down Expand Up @@ -188,7 +201,7 @@ All the namespace metrics are labeled with the following labels:
| pulsar_subscription_delayed | Gauge | The total message batches (entries) are delayed for dispatching. |
| pulsar_storage_write_latency_le_* | Histogram | The entry rate of a namespace that the storage write latency is smaller with a given threshold.<br /> Available thresholds: <br /><ul><li>pulsar_storage_write_latency_le_0_5: <= 0.5ms </li><li>pulsar_storage_write_latency_le_1: <= 1ms</li><li>pulsar_storage_write_latency_le_5: <= 5ms</li><li>pulsar_storage_write_latency_le_10: <= 10ms</li><li>pulsar_storage_write_latency_le_20: <= 20ms</li><li>pulsar_storage_write_latency_le_50: <= 50ms</li><li>pulsar_storage_write_latency_le_100: <= 100ms</li><li>pulsar_storage_write_latency_le_200: <= 200ms</li><li>pulsar_storage_write_latency_le_1000: <= 1s</li><li>pulsar_storage_write_latency_le_overflow: > 1s</li></ul> |
| pulsar_entry_size_le_* | Histogram | The entry rate of a namespace that the entry size is smaller with a given threshold.<br /> Available thresholds: <br /><ul><li>pulsar_entry_size_le_128: <= 128 bytes </li><li>pulsar_entry_size_le_512: <= 512 bytes</li><li>pulsar_entry_size_le_1_kb: <= 1 KB</li><li>pulsar_entry_size_le_2_kb: <= 2 KB</li><li>pulsar_entry_size_le_4_kb: <= 4 KB</li><li>pulsar_entry_size_le_16_kb: <= 16 KB</li><li>pulsar_entry_size_le_100_kb: <= 100 KB</li><li>pulsar_entry_size_le_1_mb: <= 1 MB</li><li>pulsar_entry_size_le_overflow: > 1 MB</li></ul> |
| pulsar_delayed_message_index_size_bytes | Gauge | The total memory size allocated by `InMemoryDelayedDeliveryTracker` of the namespace owned by this broker (in bytes). |
| pulsar_delayed_message_index_size_bytes | Gauge | The total memory size allocated by `InMemoryDelayedDeliveryTracker` of the namespace owned by this broker (in bytes). |

### Topic metrics

Expand Down Expand Up @@ -331,7 +344,7 @@ All the loadbalancing metrics are labeled with the following labels:
| pulsar_lb_bandwidth_in_usage | Gauge | The broker inbound bandwidth usage (in percent). |
| pulsar_lb_bandwidth_out_usage | Gauge | The broker outbound bandwidth usage (in percent). |
| pulsar_lb_cpu_usage | Gauge | The broker cpu usage (in percent). |
| pulsar_lb_directMemory_usage | Gauge | The broker process direct memory usage (in percent). |
| pulsar_lb_directMemory_usage | Gauge | The broker process direct memory usage (in percent). |
| pulsar_lb_memory_usage | Gauge | The broker process memory usage (in percent). |

### BundleUnloading metrics
Expand Down