Skip to content

Save raw metrics exposition in Phase 0 and mark ddev env tools read-only#23998

Open
luisorofino wants to merge 1 commit into
loa/openmetrics-ai-genfrom
loa/metrics-exposition
Open

Save raw metrics exposition in Phase 0 and mark ddev env tools read-only#23998
luisorofino wants to merge 1 commit into
loa/openmetrics-ai-genfrom
loa/metrics-exposition

Conversation

@luisorofino

@luisorofino luisorofino commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Two small, self-contained additions to the OpenMetrics generation flow's plumbing:

  • Phase 0 (inspect endpoint) now also saves the verbatim endpoint body to a <phase_id>_exposition.txt sidecar, records its path in the checkpoint, and surfaces it in the phase memory — alongside the existing parsed JSONL catalog.
  • ddev_env_show and ddev_env_test are marked read_only=True in the tool registry, since neither writes integration files.

Includes unit tests for the new sidecar (path in checkpoint, verbatim contents, memory reference, and the atomic-write failure path).

Motivation

A later phase generates the integration's unit tests, which mock the metrics endpoint from a tests/fixtures/metrics.txt capture. Persisting the raw body at inspection time gives a fixture that exactly matches the metric catalog every downstream phase works from — no re-fetching a possibly-changed endpoint, no drift from the parsed snapshot.

The read_only flags let the goal-validation reviewer (which only receives read-only tools) run ddev env show/ddev env test to objectively verify a generated integration's tests pass, instead of relying on the worker's self-report. Both tools only list environments / run tests; they never mutate integration files, so the classification is accurate. ddev_test/ddev_validate stay non-read-only because their fix/sync modes do write files.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@luisorofino luisorofino added the qa/skip-qa Automatically skip this PR for the next QA label Jun 10, 2026
@dd-octo-sts dd-octo-sts Bot added the ddev label Jun 10, 2026
@luisorofino luisorofino changed the title Add metrics exposition to inspect endpoint and mark some ddev tools a… Add metrics exposition to phase 0 Jun 10, 2026
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests  Code Coverage

Fix all issues with BitsAI

⚠️ Warnings

🚦 4 Pipeline jobs failed

PR All | test / j636396f / n8n   View in Datadog   GitHub Actions

PR All | test / j91231ff (py3.13-6.0-standalone) / MongoDB-py3.13-6.0-standalone   View in Datadog   GitHub Actions

PR All | test / ja114815 / NGINX Ingress Controller   View in Datadog   GitHub Actions

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 88.05%

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e87403d | Docs | Datadog PR Page | Give us feedback!

@luisorofino luisorofino changed the title Add metrics exposition to phase 0 Save raw metrics exposition in Phase 0 and mark ddev env tools read-only Jun 10, 2026
@luisorofino luisorofino marked this pull request as ready for review June 10, 2026 12:42
@luisorofino luisorofino requested a review from a team as a code owner June 10, 2026 12:42

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e87403d36b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

"ddev_env_start": ToolSpec("shell.ddev.env_start", "DdevEnvStartTool", read_only=False),
"ddev_env_stop": ToolSpec("shell.ddev.env_stop", "DdevEnvStopTool", read_only=False),
"ddev_env_test": ToolSpec("shell.ddev.env_test", "DdevEnvTestTool", read_only=False),
"ddev_env_test": ToolSpec("shell.ddev.env_test", "DdevEnvTestTool", read_only=True),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep ddev_env_test out of read-only tools

When a goal-reviewer is limited to read-only tools, this entry now gives it ddev_env_test, but that command is not just observational for inactive environments or environment="all": I checked ddev/src/ddev/cli/env/test.py, where it invokes start before running tests and stop in finally, and the command docstring says inactive environments are started and stopped automatically. That can create/tear down E2E containers and write environment metadata/config, so it violates the registry contract that read-only tools only inspect state and never mutate it.

Useful? React with 👍 / 👎.

@dd-octo-sts

dd-octo-sts Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ddev qa/skip-qa Automatically skip this PR for the next QA team/agent-integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants