spec.yaml

name: MongoDB
files:
- name: mongo.yaml
  options:
  - template: init_config
    options:
    - template: init_config/default
  - template: instances
    options:
    - name: hosts
      enabled: true
      description: |
        The host (and optional port number) where the mongod instance (or mongos instances for
        a sharded cluster) is running. You can specify a hostname, IP address, or UNIX domain
        socket. Specify as many hosts as appropriate for your deployment topology:

          - For a standalone, specify the hostname of the standalone mongod instance.
          - For a replica set, specify the hostname of a mongod instance as listed in
            the replica set configuration. You need one check instance per mongod instance
            in order to collect replication metrics and events.
          - For a sharded cluster, you need one check instance for each mongod instance in
            each shard (including the configsvr shard) as well as one additional check instance
            that connects to one (or more) mongos node.

        If the port number is not specified, the default port 27017 is used.

        Even if the host is an arbiter node, you still need to provide authentication credentials
        as the check will create an additional connection to the primary in this specific case.

        Do not specify more than one host here unless you are connecting to a mongos instance
        and you want to balance requests performed by the agent to multiple mongos instances.

        If you need to monitor multiple mongos instances and are using version 7.39 or later of the Agent,
        configure the integration to disable directConnection, which is enabled by default starting in 4.0.2.
        This can be done by setting options.directConnection to 'false'.

      value:
        example:
        - <HOST>:<PORT>
        anyOf:
          - type: string
          - type: array
            items:
              type: string
    - name: username
      description: |
        The username to use for authentication.
      value:
        type: string
    - name: password
      description: |
        The password to use for authentication.
      secret: true
      value:
        type: string
    - name: connection_scheme
      description: |
        Use `mongodb` to identify that this is a string in the standard connection format. This is the default value.
        In order to leverage the DNS seed list, use a connection_scheme of `mongodb+srv` rather than the standard
        `mongodb`.
        For more information, see: https://www.mongodb.com/docs/manual/reference/connection-string/
      value:
        type: string
        example: mongodb
    - name: database
      description: |
        The database to collect metrics from.
        This is also the authentication database to use if a `username` and `password` is set but the `authSource` is not
        specified in `options`. If `database` is not set and the `authSource` is not specified in `options`,
        the check will attempt to authenticate `username` to the admin database.
      value:
        type: string
    - name: options
      description: |
        For a complete list of available connection options, see:
        https://docs.mongodb.com/manual/reference/connection-string/#connections-connection-options

        NOTE: For monitoring replica set, do not use the replicaSet parameter as the Agent expects to always
        connect to the same host. Instead configure one check instance for each node.
      value:
        type: object
        properties: []
    - name: dbnames
      legacy: true
      deprecation:
        Agent version: "7.56.0"
        Migration: |
          dbnames is deprecated. Set database_autodiscovery.enabled to true to enable database autodiscovery.
          Use database_autodiscovery.include or database_autodiscovery.exclude to include or exclude
          specific databases to collect metrics from.
      description: |
        Set a list of the names of all databases to collect dbstats metrics from.
        If this key does not exist, all dbstats metrics from all databases on the server will be collected.
      value:
        type: array
        items:
          type: string
        example:
          [ one_database, other_database ]
    - name: database_autodiscovery
      description: |
        Define the configuration for database autodiscovery.
        Complete this section if you want to auto-discover databases on this MongoDB instance.
      options:
        - name: enabled
          description: Enable database autodiscovery.
          value:
            type: boolean
            example: false
            display_default: false
        - name: max_databases
          description: The maximum number of databases this host should monitor.
          value:
            type: integer
            example: 100
            display_default: 100
        - name: include
          description: |
            Regular expression for database names to include as part of
            database autodiscovery.
            Will report metrics for databases that are found in this instance,
            ignores databases listed but not found.
            Character casing is ignored. The regular expressions start matching from
            the beginning, so to match anywhere, prepend `.*`. For exact matches append `$`.
            Defaults to `.*` to include everything.
          value:
            type: array
            items:
              type: string
            example:
              - "mydatabase$"
              - "orders.*"
            display_default:
              - ".*"
        - name: exclude
          description: |
            Regular expression for database names to exclude as part of `database_autodiscovery`.
            Character casing is ignored. The regular expressions start matching from the beginning,
            so to match anywhere, prepend `.*`. For exact matches append `$`.
            In case of conflicts, database exclusion via `exclude` takes precedence over
            those found via `include`
          value:
            type: array
            items:
              type: string
            example:
              - "admin$"
              - "config$"
              - "local$"
            display_default:
              - "admin$"
              - "config$"
              - "local$"
        - name: max_collections_per_database
          description: |
            The maximum number of collections to collect metrics from per database.
            Defaults to 100.
          value:
            type: integer
            example: 100
            display_default: 100
        - name: refresh_interval
          description: Frequency in seconds of scans for new databases. Defaults to 10 minutes.
          value:
            type: integer
            example: 600
            display_default: 600
    - name: dbm
      description: |
        Set to `true` enable Database Monitoring.
      enabled: false
      value:
        type: boolean
        example: false
        display_default: false
    - name: cluster_name
      description: |
        The name of the cluster to which the monitored MongoDB instance belongs.
        Used to group MongoDB instances in a MongoDB cluster.
        Please note that the cluster name must be unique for each MongoDB cluster.

        Required when `dbm` is enabled.
      value:
        type: string
    - name: reported_database_hostname
      description: |
        Set the reported database hostname for the connected mongodb instance. This value overrides the mongodb hostname
        detected by the Agent from mongodb admin command serverStatus. It can be useful to set a custom hostname
        when connecting to a remote database through a proxy.
      value:
        type: string
    - name: database_instance_collection_interval
      hidden: true
      description: |
        Set the database instance collection interval (in seconds). The database instance collection sends
        basic information about the database instance along with a signal that it still exists.
        This collection does not involve any additional queries to the database.
      value:
        type: number
        example: 300
        display_default: false
    - name: operation_samples
      description: Configure collection of MongoDB operation samples and explain plans.
      options:
        - name: enabled
          description: |
            Enable collection of operation samples. Requires `dbm: true`.
          value:
            type: boolean
            example: true
        - name: collection_interval
          description: |
            Set the operation samples collection interval in seconds. Each collection involves capturing
            current operations with $currentOp aggregation pipeline and explain plans for each operation.
          value:
            type: number
            example: 10
        - name: explain_verbosity
          description: |
            Set the verbosity level of explain plans. The verbosity level can be either 'queryPlanner',
            'executionStats', 'allPlansExecution' or 'disabled'. The default value is 'queryPlanner'.
            Use 'executionStats' to collect the execution details of the winning plan.
            Use 'allPlansExecution' to include partial execution data captured during plan selection.
            To disable explain plans completely, set this option to 'disabled'.

            NOTE: Setting this option to 'executionStats' or 'allPlansExecution' may impact performance
            as it requires the execution of the query to collect additional execution and plan data.
          value:
            type: string
            example: queryPlanner
    - name: slow_operations
      description: Configure collection of MongoDB slow operations.
      options:
        - name: enabled
          description: |
            Enable collection of slow operations. Requires `dbm: true`.
          value:
            type: boolean
            example: true
        - name: collection_interval
          description: |
            Set the slow operations collection interval in seconds. Each collection involves capturing
            profiled slow operations from sys.profile collection or from the Slow Query log.
          value:
            type: number
            example: 10
        - name: max_operations
          description: |
            Set the maximum number of slow operations to collect per interval.
          value:
            type: number
            example: 1000
        - name: explain_verbosity
          description: |
            Set the verbosity level of explain plans. The verbosity level can be either 'queryPlanner',
            'executionStats', 'allPlansExecution' or 'disabled'. The default value is 'queryPlanner'.
            Use 'executionStats' to collect the execution details of the winning plan.
            Use 'allPlansExecution' to include partial execution data captured during plan selection.
            To disable explain plans completely, set this option to 'disabled'.

            NOTE: Setting this option to 'executionStats' or 'allPlansExecution' may impact performance
            as it requires the execution of the query to collect additional execution and plan data.
          value:
            type: string
            example: queryPlanner
    - name: collect_schemas
      description: Configure collection of MongoDB schemas (inferred) by sampling documents.
      options:
        - name: enabled
          description: |
            Enable collection of schemas. Requires `dbm: true`.
          value:
            type: boolean
            example: true
        - name: collection_interval
          description: |
            Set the schemas collection interval in seconds. Each collection involves sampling documents
            from collections and inferring schemas.
          value:
            type: number
            example: 3600
        - name: sample_size
          description: |
            Set the sample size for each collection. The sample size is the number of documents to sample
            from each collection.
            NOTE: Increasing the sample size for better schema inference accuracy may impact performance.
          value:
            type: number
            example: 10
        - name: max_collections
          description: |
            Set the maximum number of collections to collect schemas from per interval. The maxium number of collections
            per database is bounded by `database_autodiscovery.max_collections_per_database`. By setting this option, you
            are adding an additional limit to the total number of collections across all monitored databases that schemas
            will be collected from. By default, this option is not set.
          value:
            type: number
            display_default: null
            example: 300
        - name: max_depth
          description: |
            Set the maximum depth of nested documents to sample.
          value:
            type: number
            example: 5
        - name: collect_search_indexes
          description: |
            Set to `true` to collect search indexes for each collection.
            NOTE: This option is only applicable to MongoDB Atlas clusters.
          value:
            type: boolean
            example: false
    - name: schemas
      deprecation:
        Agent version: 7.69.0
        Migration: Use `collect_schemas` instead.
      description: |
        DEPRECATED: Use `collect_schemas` instead.
        Configure collection of MongoDB schemas (inferred) by sampling documents.
      options:
        - name: enabled
          description: |
            Enable collection of schemas. Requires `dbm: true`.
          value:
            type: boolean
            example: true
        - name: collection_interval
          description: |
            Set the schemas collection interval in seconds. Each collection involves sampling documents
            from collections and inferring schemas.
          value:
            type: number
            example: 3600
        - name: sample_size
          description: |
            Set the sample size for each collection. The sample size is the number of documents to sample
            from each collection.
            NOTE: Increasing the sample size for better schema inference accuracy may impact performance.
          value:
            type: number
            example: 10
        - name: max_collections
          description: |
            Set the maximum number of collections to collect schemas from per interval. The maxium number of collections
            per database is bounded by `database_autodiscovery.max_collections_per_database`. By setting this option, you
            are adding an additional limit to the total number of collections across all monitored databases that schemas
            will be collected from. By default, this option is not set.
          value:
            type: number
            display_default: null
            example: 300
        - name: max_depth
          description: |
            Set the maximum depth of nested documents to sample.
          value:
            type: number
            example: 5
        - name: collect_search_indexes
          description: |
            Set to `true` to collect search indexes for each collection.
            NOTE: This option is only applicable to MongoDB Atlas clusters.
          value:
            type: boolean
            example: false
    - name: query_metrics
      description: |
        Query metrics provide aggregated performance statistics for normalized queries.
        This feature relies on the $queryStats aggregation pipeline, which is currently in beta.
        It requires MongoDB 7.0+ for Atlas deployments or 8.0+ for self-hosted, and the clusterMonitor role.

        See MongoDB documentation for details:
        https://www.mongodb.com/docs/manual/reference/operator/aggregation/queryStats/
      options:
        - name: enabled
          description: |
            Enable collection of query metrics. Requires `dbm: true` and MongoDB 7.0+ (Atlas) or 8.0+ (self-hosted).
            Unlike other DBM features, query_metrics must be explicitly enabled.
          value:
            type: boolean
            example: false
            display_default: false
        - name: collection_interval
          description: |
            Set the query metrics collection interval in seconds.
          value:
            type: number
            example: 10
            display_default: 10
        - name: full_statement_text_cache_max_size
          description: |
            Maximum size of the cache for full query text events.
          value:
            type: number
            example: 10000
            display_default: 10000
        - name: full_statement_text_samples_per_hour_per_query
          description: |
            Maximum number of full query text events per query per hour.
          value:
            type: number
            example: 1
            display_default: 1
    - name: replica_check
      description: |
        Whether or not to read from available replicas.
        Disable this if any replicas are inaccessible to the Agent. This option is not supported for sharded clusters.
      value:
        type: boolean
        example: true
    - name: timeout
      description: |
        Controls connectTimeoutMS, serverSelectionTimeoutMS, socketTimeoutMS and timeoutMS
        see https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html
      value:
        type: integer
        example: 30
    - name: tls
      description: If `True`, create the connection to the server using transport layer security.
      value:
        type: boolean
        example: false
    - name: tls_certificate_key_file
      description: A file containing the client certificate and private key. Implies tls=True.
      value:
        type: string
    - name: tls_ca_file
      description: |
        A file containing one or a bundle of “certification authority” certificates,
        which are used to validate certificates passed from the other end of the connection.
        Implies tls=True.
      value:
        type: string
    - name: tls_allow_invalid_hostnames
      description: |
        If `True`, disables TLS hostname verification. tls_allow_invalid_hostnames=False implies tls=True.
        Think very carefully before setting this to True as that could make your application
        vulnerable to on-path attackers.
      value:
        type: boolean
        example: false
    - name: tls_allow_invalid_certificates
      description: |
        If `True`, continues the TLS handshake regardless of the outcome of the certificate verification process.
        Think very carefully before setting this to True as that could make your application
        vulnerable to on-path attackers.
      value:
        type: boolean
        example: false
    - name: dbstats_tag_dbname
      description: |
        Whether or not to include the 'dbname' as a tag for dbstat metrics
      value:
        type: boolean
        example: true
    - name: additional_metrics
      description: |
        By default, the check collects a sample of metrics from MongoDB.
        This  parameter instructs the check to collect additional metrics on specific topics.
        Available options are:
          * `metrics.commands` - Use of database commands
          * `tcmalloc` -  TCMalloc memory allocator
          * `top` - Usage statistics for each collection
          * `collection` - Metrics of the specified collections
          * `jumbo_chunks` - Count and percentage of jumbo chunks. Ignored on mongod instances.
          * `sharded_data_distribution` - Distribution of data in sharded collections.
      value:
        type: array
        items:
          type: string
        example:
          [metrics.commands, tcmalloc, top, collection, jumbo_chunks, sharded_data_distribution]
    - name: metrics_collection_interval
      description: |
        The interval in seconds at which to collect certain types of metrics.
      hidden: true
      options:
        - name: collection
          description: |
            The interval in seconds at which to collect collection metrics.
            Only applicable when `collection` is added to `additional_metrics`.
          value:
            type: integer
            example: 300
        - name: collections_indexes_stats
          description: |
            The interval in seconds at which to collect collection indexes stats metrics.
            Only applicable when `collections_indexes_stats` is set to `true`.
          value:
            type: integer
            example: 300
        - name: sharded_data_distribution
          description: |
            The interval in seconds at which to collect sharded data distribution metrics.
            Only applicable when `sharded_data_distribution` is added to `additional_metrics`.
          value:
            type: integer
            example: 300
        - name: db_stats
          description: |
            The interval in seconds at which to collect db stats metrics.
          value:
            type: integer
            example: 15
        - name: session_stats
          description: |
            The interval in seconds at which to collect session stats metrics.
          value:
            type: integer
            example: 15
    - name: collections
      description: |
        Collect metrics on specific collections from the database specified
        IT Requires `additional_metrics.collection` to be present.
        Metrics such as `mongodb.collection.count` are collected for each named collection and tagged as follows:
         * `db:<DB_NAME>` e.g. `db:<DB_NAME>`
         * `collection:<COLLECTION_NAME>` e.g. `collection:<COLLECTION_NAME>`
        Each collection generates many metrics, up to 8 + the number of indices on the collection for each collection.

        NOTE: This option is ignored when database_autodiscovery is enabled.
        Metrics are collected for all authorized collections on autodiscovered databases.
      value:
        type: array
        items:
          type: string
        example:
          - <COLLECTION_NAME>
    - name: collections_indexes_stats
      description: |
        Collect indexes access metrics for every index in every collections in
        the 'collections' list. This is available starting MongoDB 3.2.
      value:
        type: boolean
        example: false
    - name: system_database_stats
      description: |
        Enable or disable the collection of database stats, collection stats, and index stats for system databases
        (`admin`, `local`, and `config`).
        By default, this option is enabled (`true`), meaning stats for system databases are collected.
        Set to `false` to disable collection of stats for all system databases.
      value:
        type: boolean
        example: true
    - name: free_storage_metrics
      description: |
        Enabled or disable the collection of free storage metrics.
        By default, this option is enabled (`true`), meaning free storage metrics are collected.
        Set to `false` to disable collection of free storage metrics.

        Note: If the instance has a large number of collections or indexes, obtaining free space usage data may cause
        processing delays and high CPU usage on the instance.
        https://www.mongodb.com/docs/manual/reference/command/dbStats/#std-label-dbStats-freeStorage
      value:
        type: boolean
        example: true
        display_default: true
    - name: add_node_tag_to_events
      description: |
        Adds the Mongo node to events as a tag rather than creating a seperate host for the event.
      hidden: true
      value:
        type: boolean
        example: true
    - name: custom_queries
      description: |
        Define custom queries to collect custom metrics on your Mongo
        Note: Custom queries are ignored by default when the mongo node is a secondary of a replica set.
        You can add `run_on_secondary: true` for each query that you want to run on both primaries and secondaries.
        See https://docs.datadoghq.com/integrations/guide/mongo-custom-query-collection to learn more.
        Note: For MongoDB date types you can use functions matching any of these regular expressions:
        "ISODate\(\s*\'(.*?)\'\s*\)" (e.g. "ISODate('2022-12-15T08:26:00.969Z')")
        "ISODate\(\s*\)|Date\(\s*\)" (e.g. "Date()")
        "new\s*Date\(ISODate\(\s*\)\.getTime\(\s*\)((\s*[+\-*\/]\s*(\d+))*)\s*\)"
          (e.g. "new Date(ISODate().getTime() - 60 * 1000)")
      value:
        type: array
        items:
          type: object
          properties:
            - name: metric_prefix
              type: string
            - name: database
              type: string
            - name: query
              type: object
            - name: fields
              type: array
              items:
                type: object
                properties:
                  - name: field_name
                    type: string
                  - name: name
                    type: string
                  - name: type
                    type: string
            - name: tags
              type: array
              items:
                type: string
        example:
          - metric_prefix: mongo
            query: <MONGO_COMMAND_QUERY>
            database: <MONGO_DATABASE>
            fields:
              - field_name: <FIELD_NAME>
                name: <METRIC_SUFFIX>
                type: <METRIC_TYPE>
            tags:
              - test:mongodb
    - name: aws
      description: |
        This block defines the configuration for Amazon DocumentDB instances.

        Complete this section if you have installed the Datadog AWS Integration
        (https://docs.datadoghq.com/integrations/amazon_web_services) to enrich instances
        with DocumentDB integration telemetry.

        These values are only applied when `dbm: true` option is set.
      options:
        - name: instance_endpoint
          description: |
            Equal to the Endpoint.Address of the instance the agent is connecting to.
            This value is optional if the value of `host` is already configured to the instance endpoint.

            For more information on instance endpoints,
            see the AWS docs https://docs.aws.amazon.com/documentdb/latest/developerguide/API_Endpoint.html
          value:
            type: string
            example: mydocdb.c0xa1xvklawc.us-east-1.docdb.amazonaws.com
        - name: cluster_identifier
          description: |
            Equal to the Cluster identifier of the instance the agent is connecting to.
            This value is optional if the value of `cluster_name` is already configured to the cluster identifier.

            For more information on cluster identifiers,
            see the AWS docs https://docs.aws.amazon.com/documentdb/latest/developerguide/API_DBCluster.html
          value:
            type: string
            example: mydocdb
    - name: server
      legacy: true
      deprecation:
        Agent version: "8.0.0"
        Migration: |
          Use the following options instead:
          hosts, username, password, database, options
      description: |
        Specify the MongoDB URI, with database to use for reporting (defaults to "admin")
        E.g. mongodb://datadog:LnCbkX4uhpuLHSUrcayEoAZA@localhost:27016/admin
      value:
        example: mongodb://<USER>:<PASSWORD>@<HOST>:<PORT>/<DB_NAME>
        display_default: null
        type: string
    - template: instances/default
  - template: logs
    example:
    - type: file
      path: /var/log/mongodb/mongodb.log
      source: mongo
      service: <SERVICE>