Skip to content

Latest commit

 

History

History
219 lines (149 loc) · 9.79 KB

README.md

File metadata and controls

219 lines (149 loc) · 9.79 KB

RabbitMQ Check

RabbitMQ Dashboard

Overview

This check monitors RabbitMQ through the Datadog Agent. It allows you to:

  • Track queue-based stats: queue size, consumer count, unacknowledged messages, redelivered messages, and more.
  • Track node-based stats: waiting processes, used sockets, used file descriptors, and more.
  • Monitor vhosts for aliveness and number of connections.

Consider Data Streams Monitoring to enhance your RabbitMQ integration. This solution enables pipeline visualization and lag tracking, helping you identify and resolve bottlenecks.

Setup

Installation

The RabbitMQ check is included in the Datadog Agent package. No additional installation is needed on your server.

Configuration

RabbitMQ exposes metrics in two ways: the RabbitMQ Management Plugin and the RabbitMQ Prometheus Plugin. The Datadog integration supports both versions. Follow the configuration instruction in this file that pertain to the version you intend to use. The Datadog integration also comes with an out-of-the-box dashboard and monitors for each version, as labeled by the Dashboard and Monitor titles.

Prepare RabbitMQ

RabbitMQ Prometheus Plugin.

Starting with RabbitMQ v3.8, the RabbitMQ Prometheus Plugin is enabled by default.

The Prometheus plugin version of RabbitMQ requires Python 3 support by the Datadog Agent, and so can only be supported by Agent v6 or later. Please ensure your agent is updated before configuring the Prometheus plugin version of the integration.

Configure the prometheus_plugin section in your instance configuration. When using the prometheus_plugin option, settings related to the Management Plugin are ignored.

instances:
  - prometheus_plugin:
      url: http://<HOST>:15692

This enables scraping of the /metrics endpoint on one RabbitMQ node. Datadog can also collect data from the /metrics/detailed endpoint. The metrics collected will depend on which families are included.

 instances:
   - prometheus_plugin:
       url: http://<HOST>:15692
       unaggregated_endpoint: detailed?family=queue_coarse_metrics&family=queue_consumer_count&family=channel_exchange_metrics&family=channel_queue_exchange_metrics&family=node_coarse_metrics

This configuration will collect metrics for each queue, exchange, and node. See the /metrics/detailed API documentation for more information on the metrics provided by each family.

RabbitMQ Management Plugin.

Enable the plugin. The Agent user then needs at least the monitoring tag and these required permissions:

Permission Command
conf ^aliveness-test$
write ^amq\.default$
read .*

Create an Agent user for your default vhost with the following commands:

rabbitmqctl add_user datadog <SECRET>
rabbitmqctl set_permissions  -p / datadog "^aliveness-test$" "^amq\.default$" ".*"
rabbitmqctl set_user_tags datadog monitoring

Here, / refers to the default host. Set this to your specified virtual host name. See the RabbitMQ documentation for more information.

Host

To configure this check for an Agent running on a host:

Metric collection

  1. Edit the rabbitmq.d/conf.yaml file, in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your RabbitMQ metrics. See the sample rabbitmq.d/conf.yaml for all available configuration options.

    Note: The Agent checks all queues, vhosts, and nodes by default, but you can provide lists or regexes to limit this. See the rabbitmq.d/conf.yaml for examples.

  2. Restart the Agent.

Log collection

Available for Agent versions >6.0

  1. To modify the default log file location either set the RABBITMQ_LOGS environment variable or add the following to your RabbitMQ configuration file (/etc/rabbitmq/rabbitmq.conf):

      log.dir = /var/log/rabbit
  log.file = rabbit.log

  2. Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:

    logs_enabled: true

  3. Edit the logs section of your rabbitmq.d/conf.yaml file to start collecting your RabbitMQ logs:

    logs:
  - type: file
    path: /var/log/rabbit/*.log
    source: rabbitmq
    service: myservice
    log_processing_rules:
      - type: multi_line
        name: logs_starts_with_equal_sign
        pattern: "="

  4. Restart the Agent.

Containerized

You can take advantage of Datadog's Docker container Autodiscovery, see the auto_conf.yaml example configuration for RabbitMQ-specific settings.

For container environments such as Kubernetes, see the Autodiscovery Integration Templates for guidance on applying the parameters below.

Metric collection
Parameter Value
<INTEGRATION_NAME> rabbitmq
<INIT_CONFIG> blank or {}
<INSTANCE_CONFIG> {"prometheus_plugin": {"url": "http://%%host%%:15692"}}
Log collection

Available for Agent v6.0 or later

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

Parameter Value
<LOG_CONFIG> {"source": "rabbitmq", "service": "rabbitmq", "log_processing_rules": [{"type":"multi_line","name":"logs_starts_with_equal_sign", "pattern": "="}]}

Validation

Run the Agent's status subcommand and look for rabbitmq under the Checks section.

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this integration.

Events

Service Checks

See service_checks.json for a list of service checks provided by this integration.

Troubleshooting

Migrating to Prometheus Plugin

The Prometheus Plugin exposes a different set of metrics from the Management Plugin. Here is what to be aware of as you migrate from the Management to the Prometheus Plugin.

  • Look up your metrics in this table. If a metric's description contains an [OpenMetrics] tag, then it is available in the Prometheus Plugin. Metrics available only in the Management Plugin do not have any tags in their descriptions.
  • Any dashboards and monitors using Management Plugin metrics do not function. Switch to the dashboards and monitors marked as OpenMetrics Version.
  • The default configuration collects aggregated metrics. This means, for example, that there are no metrics tagged by queue. Configure the option prometheus_plugin.unaggregated_endpoint to get metrics without aggregation.
  • The rabbitmq.status service check is replaced by rabbitmq.openmetrics.health. The service check rabbitmq.aliveness has no equivalent in the Prometheus Plugin.

The Prometheus Plugin changes some tags. The table below describes the changes to the more common tags.

Management Prometheus
queue_name queue
rabbitmq_vhost vhost, exchange_vhost, queue_vhost
rabbitmq_exchange exchange

For more information, see Tagging RabbitMQ queues by tag family.

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles: