Skip to content

nginx: Add container-based config discovery support - #24118

Merged
martavicentenavarro merged 5 commits into
masterfrom
feat/martavicentenavarro/DSCVR-485-nginx-config-discovery
Jul 20, 2026
Merged

nginx: Add container-based config discovery support#24118
martavicentenavarro merged 5 commits into
masterfrom
feat/martavicentenavarro/DSCVR-485-nginx-config-discovery

Conversation

@martavicentenavarro

@martavicentenavarro martavicentenavarro commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds container-based config discovery support to the nginx integration. When the Datadog Agent detects a container with the nginx AD identifier, it probes the container's exposed ports — starting with port 80 if it is among them — until it finds the nginx status endpoint, then schedules a check against it automatically — no manual conf.yaml needed. HTTPS candidates are generated for ports 443/8443 and for ports named https/ssl/tls; everything else is probed over HTTP. This keeps the check's TLS verification defaults unchanged, so TLS-only nginx discovery is supported when the certificate is trusted by the Agent. Deployments using self-signed or otherwise untrusted certificates are not covered by this PR unless TLS verification is configured separately.

Motivation

DSCVR-509 as part of DSCVR-485. Follows the same pattern established by the krakend integration.

Implementation note — custom discovery strategy: the framework's standard registry strategies (from_ports, openmetrics_from_ports) generate candidates from a single fixed template, which can't vary the nginx_status_url scheme (http/https) per port. Since that scheme depends on which port matched, this PR defines a small custom strategy, local:from_nginx_ports (datadog_checks/nginx/config_models/discovery_strategies.py), that classifies each candidate port as HTTP or HTTPS and yields the matching scheme alongside the port.

Port selection rationale: Port 80 is nginx's default and documented HTTP port. 443/8443 are treated as HTTPS candidates so TLS-only deployments aren't left undiscoverable.

Endpoint: /nginx_status is the standard path for the ngx_http_stub_status_module, used by the integration via the nginx_status_url instance parameter.

Scope: this PR covers the open-source/standard nginx build only, discovered over HTTP or HTTPS depending on the port, and only when the Agent trusts the certificate (see the TLS note above). NGINX Plus is not covered — there's no public Docker image for it to validate against in this repo's e2e setup. Both NGINX Plus support and self-signed/untrusted-certificate discovery are left as possible follow-ups.

docker-compose change: the ${NGINX_CONFIG_FOLDER} mount now has a fallback (${NGINX_CONFIG_FOLDER:-/tmp}). Without one, docker compose ps (called internally by the new test_e2e_discovery_all_candidates test helper) fails when the env var isn't set. The fallback is /tmp (a directory, matching /etc/nginx) rather than /dev/null (a file) to keep the mount types consistent — it's never actually used to run the container, since the real config folder is always passed via env_vars= in the tests.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jun 19, 2026

Copy link
Copy Markdown

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 94.64% (+6.35%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2edb963 | Docs | Datadog PR Page | Give us feedback!

@martavicentenavarro martavicentenavarro self-assigned this Jun 19, 2026
@martavicentenavarro martavicentenavarro added the qa/skip-qa Automatically skip this PR for the next QA label Jun 19, 2026
janine-c
janine-c previously approved these changes Jun 19, 2026
martavicentenavarro and others added 2 commits June 26, 2026 14:59
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@martavicentenavarro
martavicentenavarro force-pushed the feat/martavicentenavarro/DSCVR-485-nginx-config-discovery branch from 8eba3f4 to 16c525a Compare June 26, 2026 13:01
@martavicentenavarro
martavicentenavarro requested review from a team as code owners June 26, 2026 13:01
Comment thread nginx/tests/test_e2e.py Outdated
Comment thread nginx/assets/configuration/spec.yaml Outdated
Comment thread nginx/tests/docker/docker-compose.yaml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Autodiscovery “configuration discovery” support for the nginx integration so the Agent can auto-schedule the check for containers identified as nginx, without requiring a manual conf.yaml.

Changes:

  • Add auto_conf.yaml and spec.yaml discovery stanza to enable container-based config discovery (port-hint 80, /nginx_status).
  • Add generated discovery plumbing (config_models/discovery*.py) and an e2e discovery test path.
  • Add changelog entry documenting the new discovery support.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
nginx/tests/test_e2e.py Adds e2e tests for discovery-driven scheduling and candidate stability.
nginx/tests/docker/docker-compose.yaml Adjusts nginx test container volume wiring.
nginx/tests/conftest.py Returns e2e discovery metadata so discovery runs can mount auto_conf.yaml into the Agent.
nginx/datadog_checks/nginx/data/auto_conf.yaml Adds AD template with ad_identifiers: [nginx] and discovery: {}.
nginx/datadog_checks/nginx/config_models/discovery.py Generated candidate generator for discovery (from ports, builds nginx_status_url).
nginx/datadog_checks/nginx/config_models/discovery_strategies.py Stub for custom local discovery strategies (generated).
nginx/datadog_checks/nginx/config_models/discovery_overrides.py Stub hook for overriding/adjusting generated candidates.
nginx/changelog.d/24118.added Changelog entry for the new discovery capability.
nginx/assets/configuration/spec.yaml Adds discovery strategy + adds auto_conf.yaml to the integration config spec.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread nginx/tests/docker/docker-compose.yaml
Comment thread nginx/datadog_checks/nginx/config_models/discovery_overrides.py
@martavicentenavarro
martavicentenavarro force-pushed the feat/martavicentenavarro/DSCVR-485-nginx-config-discovery branch from cb5dae9 to af3841d Compare June 30, 2026 17:01
@martavicentenavarro
martavicentenavarro marked this pull request as ready for review July 2, 2026 15:26

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9c160ebcc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread nginx/assets/configuration/spec.yaml Outdated
@vitkyrka

vitkyrka commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Putting this as draft for now until we can address the codex comment.

@vitkyrka
vitkyrka marked this pull request as draft July 3, 2026 08:29
…egeneration, and dependency floor

- Tighten E2E discovery metric assertions from at_least=0 to at_least=1
- Add discovery_overrides.py/discovery_strategies.py stub files
- Move ad_identifiers out of the spec discovery stanza and regenerate discovery.py/auto_conf.yaml
- Add test_e2e_discovery_all_candidates and fix the changelog filename to match PR #24118
- Add NGINX_CONFIG_FOLDER fallback in docker-compose so docker compose ps works without the env var set
- Bump datadog-checks-base floor to >=37.41.0, matching the version that introduced get_e2e_discovery_metadata, assert_all_discovery_candidates_stable, and candidate_ports
@martavicentenavarro
martavicentenavarro force-pushed the feat/martavicentenavarro/DSCVR-485-nginx-config-discovery branch from c9c160e to aa22e70 Compare July 16, 2026 13:19
…er plain HTTP

Replace the from_ports strategy with a local:from_nginx_ports strategy that
classifies each candidate port as HTTP or HTTPS (by number, 443/8443, or a
https/ssl/tls port name) and generates a single scheme-matched candidate per
port. This closes the gap Codex flagged: a TLS-only nginx on 443 no longer
gets an unusable http:// candidate, and an HTTP candidate is never generated
for a known HTTPS port.

Add unit tests asserting the generator never produces http://...:443 and
that port 80 is still tried before 443/8443.
@martavicentenavarro
martavicentenavarro marked this pull request as ready for review July 17, 2026 13:29
Comment thread nginx/tests/test_discovery.py
Comment thread nginx/tests/test_e2e.py
@dd-octo-sts

dd-octo-sts Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@martavicentenavarro
martavicentenavarro added this pull request to the merge queue Jul 20, 2026
Merged via the queue into master with commit 206fd05 Jul 20, 2026
76 of 79 checks passed
@martavicentenavarro
martavicentenavarro deleted the feat/martavicentenavarro/DSCVR-485-nginx-config-discovery branch July 20, 2026 13:01
@dd-octo-sts dd-octo-sts Bot added this to the 7.83.0 milestone Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants