Refresh Datadog example output

Author: RamazanKara

CHANGELOG.md

@@ -25,6 +25,8 @@
 - Added best-effort live GitHub App installation metadata collection for the GitHub Enterprise to Forgejo path.
 - Added referenced metric and tag metadata capture to the live Datadog collector.
 - Added best-effort live Datadog integration installation metadata collection.
+- Added a reproducible Datadog example refresh target and checked-in full example output.
+- Added fixture integration evidence refs and validation coverage for integration evidence.
 - Updated CI and release workflows to Node.js 24-native GitHub Actions.
 
 ## 0.1.0 - 2026-05-24

Makefile

@@ -8,8 +8,11 @@ GOFILES := $(shell find . -name '*.go' -not -path './bin/*' -not -path './dist/*
 PLATFORMS ?= linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64
 GOLANGCI_LINT_VERSION ?= v2.12.2
 GOLANGCI_LINT ?= bin/golangci-lint
+EXAMPLE_INPUT ?= examples/datadog-to-grafana/input/datadog-fixture.json
+EXAMPLE_DIR ?= examples/datadog-to-grafana/output
+EXAMPLE_BUNDLE ?= examples/datadog-to-grafana/openexit-example.zip
 
-.PHONY: build test fmt fmt-check lint golangci-lint smoke verify release-dist clean
+.PHONY: build test fmt fmt-check lint golangci-lint smoke example example-smoke verify release-dist clean
 
 build:
 	mkdir -p $(dir $(BINARY))
@@ -70,7 +73,37 @@ smoke:
 	$(BINARY) validate --project "$$tmp/ai-demo"; \
 	$(BINARY) export --project "$$tmp/ai-demo" --format zip --out "$$tmp/ai-demo.zip"
 
-verify: lint test build smoke
+example: build
+	rm -rf $(EXAMPLE_DIR)/openexit.yaml \
+		$(EXAMPLE_DIR)/inventory \
+		$(EXAMPLE_DIR)/assessment \
+		$(EXAMPLE_DIR)/mapping \
+		$(EXAMPLE_DIR)/generated-config \
+		$(EXAMPLE_DIR)/evidence \
+		$(EXAMPLE_DIR)/validation \
+		$(EXAMPLE_BUNDLE)
+	mkdir -p $(EXAMPLE_DIR)
+	$(BINARY) init $(EXAMPLE_DIR) --source datadog --target grafana-lgtm
+	$(BINARY) collect fixture --project $(EXAMPLE_DIR) --input $(EXAMPLE_INPUT)
+	$(BINARY) assess --project $(EXAMPLE_DIR) --target grafana-lgtm
+	$(BINARY) map --project $(EXAMPLE_DIR)
+	$(BINARY) generate --project $(EXAMPLE_DIR) --all
+	$(BINARY) validate --project $(EXAMPLE_DIR)
+	$(BINARY) export --project $(EXAMPLE_DIR) --format zip --out $(EXAMPLE_BUNDLE)
+
+example-smoke: build
+	tmp=$$(mktemp -d); \
+	trap 'rm -rf "$$tmp"' EXIT; \
+	$(BINARY) init "$$tmp/example" --source datadog --target grafana-lgtm; \
+	$(BINARY) collect fixture --project "$$tmp/example" --input ./$(EXAMPLE_INPUT); \
+	$(BINARY) assess --project "$$tmp/example" --target grafana-lgtm; \
+	$(BINARY) map --project "$$tmp/example"; \
+	$(BINARY) generate --project "$$tmp/example" --all; \
+	$(BINARY) validate --project "$$tmp/example"; \
+	$(BINARY) export --project "$$tmp/example" --format zip --out "$$tmp/openexit-example.zip"; \
+	test -s "$$tmp/openexit-example.zip"
+
+verify: lint test build smoke example-smoke
 
 release-dist:
 	rm -rf dist

README.md

@@ -47,6 +47,12 @@ make release-dist VERSION=0.1.0
 
 This writes OS/architecture binaries and `dist/SHA256SUMS`.
 
+Refresh the checked-in Datadog example project with:
+
+```bash
+make example VERSION=0.1.0-dev
+```
+
 ## Commands
 
 - `openexit version`
@@ -148,7 +154,7 @@ The assessment engine includes dashboard, monitor, SLO, cost, scale, identity, e
 
 ## Release Process
 
-The release checklist lives in `docs/release.md`. A release build should pass `make verify`, `make release-dist VERSION=0.1.0`, the Datadog definition-of-done pipeline, and validation/export for every supported assessment path.
+The release checklist lives in `docs/release.md`. A release build should pass `make verify`, `make release-dist VERSION=0.1.0`, `make example VERSION=0.1.0-dev`, the Datadog definition-of-done pipeline, and validation/export for every supported assessment path.
 
 ## Assessment Paths
 

docs/cli.md

@@ -25,6 +25,8 @@ openexit export --project ./demo --format zip --out ./openexit-demo.zip
 
 The CI test suite also runs this definition-of-done pipeline against the fixture inventory and checks that the generated project layout and export bundle contain the expected artifacts. Bundle checksums are verified against the archived file bytes.
 
+The checked-in Datadog example can be refreshed with `make example VERSION=0.1.0-dev`. CI also runs `make example-smoke` as part of `make verify` to ensure the example fixture still completes the full pipeline.
+
 `openexit validate` performs typed consistency checks, embedded JSON Schema validation, Grafana dashboard candidate validation, Prometheus alert-rule candidate validation, OpenTelemetry collector candidate validation, ArgoCD candidate validation, YAML/JSON parse checks, evidence reference checks, secret scanning, and optional external tool checks when `promtool` or `kubeconform` are installed.
 
 `openexit export` refuses to package symlinks from exported project sections, even with `--force`, so evidence bundles cannot accidentally include files from outside the project tree.

docs/release.md

@@ -36,6 +36,7 @@ The AI provider path is complete for local fixture assessment workflows and incl
 - [ ] `make verify VERSION=0.1.0` passes, including CLI smoke pipelines.
 - [ ] `make lint` runs `gofmt`, `golangci-lint`, and `go vet`.
 - [ ] `make release-dist VERSION=0.1.0` produces binaries and `dist/SHA256SUMS`.
+- [ ] `make example VERSION=0.1.0-dev` refreshes `examples/datadog-to-grafana/output/` and exports `examples/datadog-to-grafana/openexit-example.zip`.
 - [ ] Datadog definition-of-done pipeline passes:
   `init`, `collect fixture`, `assess`, `map`, `generate --all`, `validate`, `export`.
 - [ ] GitHub, Okta, Auth0, Cloudflare, Akamai, OpenAI, and Anthropic fixture/live-collector test coverage pass, and supported fixture provider pipelines validate.

examples/datadog-to-grafana/README.md

@@ -5,14 +5,9 @@ This example uses a small redacted Datadog-like fixture and writes all generated
 Run from the repository root:
 
 ```bash
-make build VERSION=0.1.0-dev
-./bin/openexit init ./demo
-./bin/openexit collect fixture --project ./demo --input ./examples/datadog-to-grafana/input/datadog-fixture.json
-./bin/openexit assess --project ./demo --target grafana-lgtm
-./bin/openexit map --project ./demo
-./bin/openexit generate --project ./demo --all
-./bin/openexit validate --project ./demo
-./bin/openexit export --project ./demo --format zip --out ./openexit-demo.zip
+make example VERSION=0.1.0-dev
 ```
 
+This refreshes `examples/datadog-to-grafana/output/` and writes an ignored bundle to `examples/datadog-to-grafana/openexit-example.zip`.
+
 The generated files are candidates only. Review every dashboard, alert rule, collector sketch, migration-plan phase gate, and runbook before operational use.

examples/datadog-to-grafana/input/datadog-fixture.json

@@ -2,25 +2,70 @@
   "site": "datadoghq.eu",
   "dashboards": [
     {
-      "id": "example-dashboard",
-      "title": "Example API Overview",
-      "tags": ["team:example"],
+      "id": "abc-123",
+      "title": "Production API Overview",
+      "url": "https://app.datadoghq.eu/dashboard/abc-123",
+      "tags": ["env:prod", "team:platform"],
       "widgets": [
-        { "id": "w1", "type": "timeseries", "title": "Requests", "query": "sum:example.requests{env:prod}.as_count()" }
-      ]
+        {
+          "id": "w1",
+          "type": "timeseries",
+          "title": "CPU by service",
+          "query": "avg:system.cpu.user{env:prod} by {service}"
+        },
+        {
+          "id": "w2",
+          "type": "note",
+          "title": "Migration notes"
+        }
+      ],
+      "queries": ["sum:trace.http.request.errors{env:prod}.as_count()"]
     }
   ],
   "monitors": [
     {
-      "id": "example-monitor",
-      "name": "Example request errors",
+      "id": "123456",
+      "name": "High error rate",
       "type": "query alert",
-      "query": "sum(last_5m):sum:example.errors{env:prod}.as_count() > 1",
-      "notificationTargets": ["slack-example"],
-      "tags": ["team:example"]
+      "query": "sum(last_5m):avg:trace.http.request.errors{env:prod} > 10",
+      "message": "Investigate API errors @slack-platform",
+      "notificationTargets": ["slack-platform"],
+      "tags": ["env:prod", "team:platform"],
+      "runbookUrl": "https://runbooks.example.invalid/high-error-rate"
+    },
+    {
+      "id": "789012",
+      "name": "Latency anomaly",
+      "type": "query alert",
+      "query": "avg(last_15m):anomalies(avg:trace.http.request.duration{env:prod}, 'basic', 2) >= 1",
+      "message": "Latency anomaly @webhook-custom",
+      "notificationTargets": ["webhook-custom"],
+      "tags": ["env:prod"]
+    }
+  ],
+  "slos": [
+    {
+      "id": "slo-123",
+      "name": "API availability",
+      "target": 99.9,
+      "timeframe": "30d"
     }
   ],
-  "slos": [],
-  "integrations": [],
-  "metrics": []
+  "integrations": [
+    {
+      "name": "kubernetes",
+      "enabled": true,
+      "tags": ["env:prod"]
+    }
+  ],
+  "metrics": [
+    {
+      "name": "system.cpu.user",
+      "tags": ["env", "service"]
+    },
+    {
+      "name": "trace.http.request.errors",
+      "tags": ["env", "service"]
+    }
+  ]
 }

examples/datadog-to-grafana/output/README.md

@@ -1,3 +1,11 @@
 # Example Output
 
-Run the commands in `examples/datadog-to-grafana/README.md` from the repository root to generate a complete example project under `./demo`.
+This directory is a checked-in OpenExit project snapshot generated from `../input/datadog-fixture.json`.
+
+Refresh it from the repository root:
+
+```bash
+make example VERSION=0.1.0-dev
+```
+
+The generated candidates are review artifacts only. Do not import dashboards, alert rules, collector config, or GitOps manifests into production without owner review, shadowing, and rollback approval.

examples/datadog-to-grafana/output/assessment/acceptance-criteria.md

@@ -0,0 +1,21 @@
+# Acceptance Criteria
+
+- Project: output
+- Source: datadog
+- Target: grafana-lgtm
+- Generated: 2026-05-27T04:39:07Z
+- Complexity: high (80)
+
+## Assumptions
+- OpenExit is local-first and performs no production writes.
+- Generated target files are candidates and require human review.
+- Evidence references point to redacted local source snapshots.
+
+## Acceptance Criteria
+- All candidate dashboards are reviewed by an owner.
+- All alert candidates shadow production Datadog alerts before cutover.
+- SLO semantics, burn-rate behavior, and notification routes are signed off.
+- Validation passes with no critical findings.
+
+## Generated by OpenExit
+This artifact was generated deterministically from local OpenExit inventory and assessment manifests. It is not production-ready until reviewed.

examples/datadog-to-grafana/output/assessment/alert-shadowing-plan.md

@@ -0,0 +1,21 @@
+# Alert Shadowing Plan
+
+- Project: output
+- Source: datadog
+- Target: grafana-lgtm
+- Generated: 2026-05-27T04:39:07Z
+- Complexity: high (80)
+
+## Assumptions
+- OpenExit is local-first and performs no production writes.
+- Generated target files are candidates and require human review.
+- Evidence references point to redacted local source snapshots.
+
+## Alert Shadowing Plan
+- Run target alert candidates without production paging.
+- Compare Datadog and target alert state over at least one representative period.
+- Investigate every high-severity finding before enabling notifications.
+- Document acceptable differences and owner sign-off.
+
+## Generated by OpenExit
+This artifact was generated deterministically from local OpenExit inventory and assessment manifests. It is not production-ready until reviewed.