-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add Config Discovery support for Vault #24851
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
martavicentenavarro
wants to merge
5
commits into
master
Choose a base branch
from
add-vault-config-discovery
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
edd650f
Add Config Discovery support for Vault
martavicentenavarro 4ef4aae
Add changelog entry for vault discovery support
martavicentenavarro 0f58c22
Try both metric-scraping candidates before either health-only fallback
martavicentenavarro c3377fa
Bump datadog-checks-base floor to 38.0.0 for vault
martavicentenavarro 68a3d5b
Remove credential-less discovery candidates for vault
martavicentenavarro File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Add Config Discovery support. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # (C) Datadog, Inc. 2026-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
|
||
| # This file is autogenerated. | ||
| # To change this file you should edit assets/configuration/spec.yaml and then run the following commands: | ||
| # ddev -x validate config -s <INTEGRATION_NAME> | ||
| # ddev -x validate models -s <INTEGRATION_NAME> | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from collections.abc import Iterator | ||
| from typing import Any | ||
|
|
||
| from datadog_checks.base.utils.discovery import Service, candidate_ports | ||
| from datadog_checks.vault.config_models import discovery_overrides | ||
| from datadog_checks.vault.config_models.instance import InstanceConfig | ||
| from datadog_checks.vault.config_models.shared import SharedConfig | ||
|
|
||
|
|
||
| def _generated_candidates(service: Service) -> Iterator[dict[str, Any]]: | ||
| shared = SharedConfig.model_validate({}, context={'configured_fields': frozenset()}).model_dump( | ||
| by_alias=True, mode='json', exclude_none=True | ||
| ) | ||
| # discovery[0]: from_ports | ||
| for port in candidate_ports(service, [8200]): | ||
| ctx = {'port': port} | ||
| instance_data = { | ||
| 'use_openmetrics': 'true', | ||
| 'api_url': 'http://{service.host}:{port.number}/v1'.format(service=service, **ctx), | ||
| 'no_token': 'true', | ||
| } | ||
| instance = InstanceConfig.model_validate( | ||
| instance_data, context={'configured_fields': frozenset(instance_data)} | ||
| ).model_dump(by_alias=True, mode='json', exclude_none=True) | ||
| yield {'init_config': shared, 'instances': [instance]} | ||
| instance_data = { | ||
| 'use_openmetrics': 'true', | ||
| 'api_url': 'http://{service.host}:{port.number}/v1'.format(service=service, **ctx), | ||
| } | ||
| instance = InstanceConfig.model_validate( | ||
| instance_data, context={'configured_fields': frozenset(instance_data)} | ||
| ).model_dump(by_alias=True, mode='json', exclude_none=True) | ||
| yield {'init_config': shared, 'instances': [instance]} | ||
| instance_data = { | ||
| 'api_url': 'http://{service.host}:{port.number}/v1'.format(service=service, **ctx), | ||
| 'no_token': 'true', | ||
| } | ||
| instance = InstanceConfig.model_validate( | ||
| instance_data, context={'configured_fields': frozenset(instance_data)} | ||
| ).model_dump(by_alias=True, mode='json', exclude_none=True) | ||
| yield {'init_config': shared, 'instances': [instance]} | ||
| instance_data = { | ||
| 'api_url': 'http://{service.host}:{port.number}/v1'.format(service=service, **ctx), | ||
| } | ||
| instance = InstanceConfig.model_validate( | ||
| instance_data, context={'configured_fields': frozenset(instance_data)} | ||
| ).model_dump(by_alias=True, mode='json', exclude_none=True) | ||
| yield {'init_config': shared, 'instances': [instance]} | ||
|
|
||
|
|
||
| def candidates(service: Service) -> Iterator[dict[str, Any]]: | ||
| override = getattr(discovery_overrides, 'candidates', None) | ||
| if override is None: | ||
| yield from _generated_candidates(service) | ||
| else: | ||
| yield from override(service, default=_generated_candidates) |
12 changes: 12 additions & 0 deletions
12
vault/datadog_checks/vault/config_models/discovery_overrides.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # (C) Datadog, Inc. 2026-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
|
||
| # Override the generated discovery candidates() for this integration. | ||
| # | ||
| # Define a candidates(service, default) function to wrap or replace the generated | ||
| # candidate generation. `default` is the generated generator; call it to reuse | ||
| # the spec-driven candidates, or ignore it to replace them entirely. | ||
| # | ||
| # def candidates(service, default): | ||
| # yield from default(service) |
18 changes: 18 additions & 0 deletions
18
vault/datadog_checks/vault/config_models/discovery_strategies.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # (C) Datadog, Inc. 2026-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
|
||
| # Here you can define custom (local:) discovery strategies for this integration. | ||
| # | ||
| # Decorate a generator with @discovery_strategy (imported from | ||
| # datadog_checks.base.utils.discovery) and reference it from the spec discovery | ||
| # stanza as `strategy: local:<function_name>`. The function receives the | ||
| # discovered Service plus the inputs declared in the spec and yields one context | ||
| # (ctx) mapping per candidate, exposing the keys listed in `provides`. | ||
| # | ||
| # from datadog_checks.base.utils.discovery import discovery_strategy | ||
| # | ||
| # @discovery_strategy(provides=('svc',)) | ||
| # def from_some_config(service, config_path): | ||
| # ... | ||
| # yield {'svc': ...} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| ## @param ad_identifiers - list of strings - required | ||
| ## A list of container identifiers that are used by Autodiscovery to identify | ||
| ## which container the check should be run against. For more information, see: | ||
| ## https://docs.datadoghq.com/agent/guide/ad_identifiers/ | ||
| # | ||
| ad_identifiers: | ||
| - vault | ||
|
|
||
| ## Enables configuration discovery | ||
| # | ||
| discovery: {} | ||
|
|
||
| ## Unused init configuration | ||
| # | ||
| init_config: | ||
|
|
||
| ## Unused instance configuration | ||
| # | ||
| instances: [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # (C) Datadog, Inc. 2026-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
| import pytest | ||
|
|
||
| from datadog_checks.base.utils.discovery import Port, Service | ||
| from datadog_checks.vault import Vault | ||
|
|
||
| pytestmark = [pytest.mark.unit] | ||
|
|
||
|
|
||
| def generated_instances(service: Service) -> list[dict]: | ||
| return [config['instances'][0] for config in Vault.generate_configs(service)] | ||
|
|
||
|
|
||
| def test_generates_one_candidate_per_mode_and_token_strategy() -> None: | ||
| # Order matters: discovery accepts the first candidate whose real check run collects a | ||
| # metric, so the richest (full OpenMetrics scrape) candidates must be tried before the | ||
| # safe fallbacks that only ever hit the always-unauthenticated leader/health endpoints. | ||
| service = Service(id='vault', host='127.0.0.1', ports=(Port(number=8200),)) | ||
|
|
||
| instances = generated_instances(service) | ||
|
|
||
| assert [(instance.get('use_openmetrics'), instance.get('no_token')) for instance in instances] == [ | ||
| (True, True), | ||
| (True, False), | ||
| (False, True), | ||
| (False, False), | ||
| ] | ||
|
|
||
|
|
||
| def test_all_candidates_target_the_same_api_url() -> None: | ||
| service = Service(id='vault', host='127.0.0.1', ports=(Port(number=8200),)) | ||
|
|
||
| instances = generated_instances(service) | ||
|
|
||
| assert all(instance['api_url'] == 'http://127.0.0.1:8200/v1' for instance in instances) | ||
|
|
||
|
|
||
| def test_ipv6_host_is_bracketed_in_generated_api_url() -> None: | ||
| service = Service(id='vault', host='fd00::1', ports=(Port(number=8200),)) | ||
|
|
||
| instances = generated_instances(service) | ||
|
|
||
| assert all(instance['api_url'] == 'http://[fd00::1]:8200/v1' for instance in instances) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a Vault listener has TLS enabled, every generated candidate uses plaintext
http://, so each probe fails before it can collect the leader/health metric and discovery returns no configuration. This excludes secure production Vault deployments even though the check'sapi_urlsupports HTTPS; include HTTPS candidates or derive the listener scheme rather than only probing HTTP.Useful? React with 👍 / 👎.