Skip to content

Commit 250cc82

Browse files
committed
Add collected project workflow runner
1 parent b624e25 commit 250cc82

7 files changed

Lines changed: 182 additions & 85 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
- Added edge VCL, HAProxy, and Coraza candidate validation for the Cloudflare/Akamai assessment path.
3333
- Added LiteLLM/vLLM config candidate validation for the OpenAI/Anthropic assessment path.
3434
- Added project readiness status with pipeline summaries, export readiness, next actions, and `--json` output.
35+
- Added `openexit run` to execute assessment, mapping, full generation, validation, status reporting, and optional evidence bundle export for collected projects.
3536
- Updated CI and release workflows to Node.js 24-native GitHub Actions.
3637

3738
## 0.1.0 - 2026-05-24

README.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,7 @@ The Datadog to Grafana LGTM path includes both fixture import and a read-only li
2222
make build
2323
./bin/openexit init ./demo --source datadog --target grafana-lgtm
2424
./bin/openexit collect fixture --project ./demo --input ./testdata/datadog/small.json
25-
./bin/openexit assess --project ./demo --target grafana-lgtm
26-
./bin/openexit map --project ./demo
27-
./bin/openexit generate --project ./demo --all
28-
./bin/openexit validate --project ./demo
29-
./bin/openexit export --project ./demo --format zip --out ./openexit-demo.zip
25+
./bin/openexit run --project ./demo --export --out ./openexit-demo.zip
3026
```
3127

3228
## Install From Source
@@ -58,6 +54,7 @@ make example VERSION=0.1.0-dev
5854
- `openexit version`
5955
- `openexit init <project-dir> [--source <type> --target <type>]`
6056
- `openexit status --project <project-dir> [--json]`
57+
- `openexit run --project <project-dir> [--strict] [--export --out <file>]`
6158
- `openexit collect fixture --project <project-dir> --input <file>`
6259
- `openexit collect github --project <project-dir> --owner <org> [--base-url https://github.example.com/api/v3] [--token-env GITHUB_TOKEN] [--repo owner/name]`
6360
- `openexit collect github-fixture --project <project-dir> --input <file>`
@@ -99,6 +96,7 @@ Included in the current implementation:
9996

10097
- CLI skeleton and project init/status.
10198
- Project readiness status with pipeline counts, validation state, export readiness, and JSON output for automation.
99+
- One-command deterministic workflow runner for collected projects, with optional evidence bundle export.
102100
- Typed project, inventory, assessment, mapping, and validation manifests.
103101
- Fixture-based Datadog inventory import.
104102
- Read-only Datadog collection for dashboards, monitors, SLOs, installed integration metadata, and referenced metric/tag metadata.

docs/cli.md

Lines changed: 14 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -16,17 +16,15 @@ The minimum local demo is:
1616
```bash
1717
openexit init ./demo --source datadog --target grafana-lgtm
1818
openexit collect fixture --project ./demo --input ./testdata/datadog/small.json
19-
openexit assess --project ./demo --target grafana-lgtm
20-
openexit map --project ./demo
21-
openexit generate --project ./demo --all
22-
openexit validate --project ./demo
23-
openexit export --project ./demo --format zip --out ./openexit-demo.zip
19+
openexit run --project ./demo --export --out ./openexit-demo.zip
2420
```
2521

2622
The CI test suite also runs this definition-of-done pipeline against the fixture inventory and checks that the generated project layout and export bundle contain the expected artifacts. Bundle checksums are verified against the archived file bytes.
2723

2824
The checked-in Datadog example can be refreshed with `make example VERSION=0.1.0-dev`. CI also runs `make example-smoke` as part of `make verify` to ensure the example fixture still completes the full pipeline.
2925

26+
`openexit run --project <project-dir>` is the ergonomic path after collection. It runs assessment, mapping, full artifact generation, validation, and final status reporting. Add `--export --out <file>` to write the evidence bundle after validation passes, or `--strict` to treat validation warnings as failures.
27+
3028
`openexit status --project <project-dir>` summarizes the current pipeline state: project layout, source/target pair, inventory counts, assessment finding severity, mapping counts, generated candidate artifacts, validation check totals, export readiness, and the next recommended command. Use `--json` to feed the same readiness data into automation or release gates.
3129

3230
`openexit validate` performs typed consistency checks, embedded JSON Schema validation, Grafana dashboard candidate validation, Prometheus alert-rule candidate validation, OpenTelemetry collector candidate validation, ArgoCD candidate validation, Forgejo migration candidate validation, identity realm/client candidate validation, edge VCL/HAProxy/Coraza candidate validation, LiteLLM/vLLM candidate validation, YAML/JSON parse checks, evidence reference checks, secret scanning, and optional external tool checks when `promtool` or `kubeconform` are installed.
@@ -40,10 +38,7 @@ The GitHub Enterprise to Forgejo fixture path uses local JSON metadata:
4038
```bash
4139
openexit init ./ghe-demo --source github-enterprise --target forgejo
4240
openexit collect github-fixture --project ./ghe-demo --input ./testdata/github-enterprise/small.json
43-
openexit assess --project ./ghe-demo --target forgejo
44-
openexit map --project ./ghe-demo
45-
openexit generate --project ./ghe-demo --all
46-
openexit validate --project ./ghe-demo
41+
openexit run --project ./ghe-demo
4742
```
4843

4944
The same path can collect read-only live GitHub or GitHub Enterprise metadata. Set the token in an environment variable; OpenExit reads it at runtime and does not write it into project files. Organization GitHub App installation metadata is collected when the token can read organization administration metadata; otherwise OpenExit records a warning and continues collecting repository-scoped metadata.
@@ -52,10 +47,7 @@ The same path can collect read-only live GitHub or GitHub Enterprise metadata. S
5247
export GITHUB_TOKEN=<read-only-token>
5348
openexit init ./ghe-live --source github-enterprise --target forgejo
5449
openexit collect github --project ./ghe-live --owner acme --token-env GITHUB_TOKEN
55-
openexit assess --project ./ghe-live --target forgejo
56-
openexit map --project ./ghe-live
57-
openexit generate --project ./ghe-live --all
58-
openexit validate --project ./ghe-live
50+
openexit run --project ./ghe-live
5951
```
6052

6153
For GitHub Enterprise Server, pass the API root:
@@ -75,10 +67,7 @@ The Okta/Auth0 to Keycloak/Zitadel fixture path uses local JSON metadata:
7567
```bash
7668
openexit init ./identity-demo --source identity --target keycloak-zitadel
7769
openexit collect identity-fixture --project ./identity-demo --input ./testdata/identity/small.json
78-
openexit assess --project ./identity-demo --target keycloak-zitadel
79-
openexit map --project ./identity-demo
80-
openexit generate --project ./identity-demo --all
81-
openexit validate --project ./identity-demo
70+
openexit run --project ./identity-demo
8271
```
8372

8473
The same path can collect read-only live Okta metadata. Set the token in an environment variable; OpenExit reads it at runtime and does not write it into project files.
@@ -87,10 +76,7 @@ The same path can collect read-only live Okta metadata. Set the token in an envi
8776
export OKTA_API_TOKEN=<read-only-token>
8877
openexit init ./okta-live --source identity --target keycloak-zitadel
8978
openexit collect okta --project ./okta-live --org-url https://dev-123456.okta.com --token-env OKTA_API_TOKEN
90-
openexit assess --project ./okta-live --target keycloak-zitadel
91-
openexit map --project ./okta-live
92-
openexit generate --project ./okta-live --all
93-
openexit validate --project ./okta-live
79+
openexit run --project ./okta-live
9480
```
9581

9682
Use repeatable `--break-glass-user` flags to verify named emergency accounts and capture whether they have active factors:
@@ -105,10 +91,7 @@ The same identity path can collect read-only live Auth0 metadata. Set an Auth0 M
10591
export AUTH0_MANAGEMENT_TOKEN=<read-only-management-token>
10692
openexit init ./auth0-live --source identity --target keycloak-zitadel
10793
openexit collect auth0 --project ./auth0-live --domain https://example.us.auth0.com --token-env AUTH0_MANAGEMENT_TOKEN
108-
openexit assess --project ./auth0-live --target keycloak-zitadel
109-
openexit map --project ./auth0-live
110-
openexit generate --project ./auth0-live --all
111-
openexit validate --project ./auth0-live
94+
openexit run --project ./auth0-live
11295
```
11396

11497
Use repeatable `--break-glass-user` flags with an email, username, or Auth0 user ID to verify emergency accounts:
@@ -122,10 +105,7 @@ The Cloudflare/Akamai to Varnish/HAProxy/Coraza fixture path uses local JSON met
122105
```bash
123106
openexit init ./edge-demo --source edge --target varnish-haproxy-coraza
124107
openexit collect edge-fixture --project ./edge-demo --input ./testdata/edge/small.json
125-
openexit assess --project ./edge-demo --target varnish-haproxy-coraza
126-
openexit map --project ./edge-demo
127-
openexit generate --project ./edge-demo --all
128-
openexit validate --project ./edge-demo
108+
openexit run --project ./edge-demo
129109
```
130110

131111
The same path can collect read-only live Cloudflare metadata. Set the API token in an environment variable; OpenExit reads it at runtime and does not write it into project files.
@@ -134,10 +114,7 @@ The same path can collect read-only live Cloudflare metadata. Set the API token
134114
export CLOUDFLARE_API_TOKEN=<read-only-token>
135115
openexit init ./cloudflare-live --source edge --target varnish-haproxy-coraza
136116
openexit collect cloudflare --project ./cloudflare-live --zone-id <zone-id> --token-env CLOUDFLARE_API_TOKEN
137-
openexit assess --project ./cloudflare-live --target varnish-haproxy-coraza
138-
openexit map --project ./cloudflare-live
139-
openexit generate --project ./cloudflare-live --all
140-
openexit validate --project ./cloudflare-live
117+
openexit run --project ./cloudflare-live
141118
```
142119

143120
The same path can collect read-only live Akamai metadata. The collector reads EdgeGrid credentials from `~/.edgerc` or the `AKAMAI_HOST`, `AKAMAI_CLIENT_TOKEN`, `AKAMAI_ACCESS_TOKEN`, and `AKAMAI_CLIENT_SECRET` environment variables. It records Edge DNS recordsets, Property Manager hostnames and rules, and optional AppSec custom-rule metadata without storing credential values.
@@ -151,10 +128,7 @@ openexit collect akamai \
151128
--contract-id ctr_1-ABCDEF \
152129
--group-id grp_12345 \
153130
--security-config-id 12345:7
154-
openexit assess --project ./akamai-live --target varnish-haproxy-coraza
155-
openexit map --project ./akamai-live
156-
openexit generate --project ./akamai-live --all
157-
openexit validate --project ./akamai-live
131+
openexit run --project ./akamai-live
158132
```
159133

160134
Use repeatable `--zone`, `--property-id`, and `--security-config-id` flags to scope collection. Use `--discover-properties` with `--contract-id` and `--group-id` to list accessible Property Manager properties before collection, and `--account-switch-key` when the API client needs to act against another account.
@@ -164,10 +138,7 @@ The OpenAI/Anthropic to vLLM/LiteLLM fixture path uses local JSON metadata:
164138
```bash
165139
openexit init ./ai-demo --source ai-provider --target vllm-litellm
166140
openexit collect ai-fixture --project ./ai-demo --input ./testdata/ai-provider/small.json
167-
openexit assess --project ./ai-demo --target vllm-litellm
168-
openexit map --project ./ai-demo
169-
openexit generate --project ./ai-demo --all
170-
openexit validate --project ./ai-demo
141+
openexit run --project ./ai-demo
171142
```
172143

173144
The same path can collect read-only aggregate OpenAI usage. Set an OpenAI admin key in an environment variable; OpenExit reads it at runtime and does not write it into project files or evidence. The live collector records model-grouped token usage and available model metadata, not raw prompts or responses.
@@ -182,10 +153,7 @@ openexit collect openai \
182153
--owner platform-ai \
183154
--fallback-strategy manual-queue \
184155
--fallback-manual-queue
185-
openexit assess --project ./openai-live --target vllm-litellm
186-
openexit map --project ./openai-live
187-
openexit generate --project ./openai-live --all
188-
openexit validate --project ./openai-live
156+
openexit run --project ./openai-live
189157
```
190158

191159
Use `--days` to change the aggregate usage window and `--peak-days` to change the hourly peak-estimate window. Use `--organization-id` or `--project-id` only when your OpenAI account requires those headers.
@@ -203,10 +171,7 @@ openexit collect anthropic \
203171
--owner platform-ai \
204172
--fallback-strategy manual-queue \
205173
--fallback-manual-queue
206-
openexit assess --project ./anthropic-live --target vllm-litellm
207-
openexit map --project ./anthropic-live
208-
openexit generate --project ./anthropic-live --all
209-
openexit validate --project ./anthropic-live
174+
openexit run --project ./anthropic-live
210175
```
211176

212177
Use `--api-key-id`, `--workspace-id`, or `--model` to restrict the Anthropic usage query. Use `--days` to change the daily aggregate usage window and `--peak-days` to change the hourly peak-estimate window.

docs/openai-anthropic-to-vllm-litellm.md

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -58,9 +58,7 @@ API surface used:
5858
```bash
5959
openexit init ./ai-demo --source ai-provider --target vllm-litellm
6060
openexit collect ai-fixture --project ./ai-demo --input ./testdata/ai-provider/small.json
61-
openexit assess --project ./ai-demo --target vllm-litellm
62-
openexit generate --project ./ai-demo --all
63-
openexit validate --project ./ai-demo
61+
openexit run --project ./ai-demo
6462
```
6563

6664
## Live OpenAI Demo
@@ -75,9 +73,7 @@ openexit collect openai \
7573
--owner platform-ai \
7674
--fallback-strategy manual-queue \
7775
--fallback-manual-queue
78-
openexit assess --project ./openai-live --target vllm-litellm
79-
openexit generate --project ./openai-live --all
80-
openexit validate --project ./openai-live
76+
openexit run --project ./openai-live
8177
```
8278

8379
## Live Anthropic Demo
@@ -93,9 +89,7 @@ openexit collect anthropic \
9389
--owner platform-ai \
9490
--fallback-strategy manual-queue \
9591
--fallback-manual-queue
96-
openexit assess --project ./anthropic-live --target vllm-litellm
97-
openexit generate --project ./anthropic-live --all
98-
openexit validate --project ./anthropic-live
92+
openexit run --project ./anthropic-live
9993
```
10094

10195
## Generated Artifacts

docs/release.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ The implementation plan names Datadog to Grafana LGTM, Prometheus-compatible ale
88

99
Release-blocking v0.1 requirements:
1010

11-
- CLI skeleton, project init, readiness status, version command.
11+
- CLI skeleton, project init, workflow runner, readiness status, version command.
1212
- Datadog fixture collector and read-only live Datadog collector.
1313
- Inventory and assessment manifests with typed validation.
1414
- Source-to-target mapping manifest with candidate paths and manual-review entries.
@@ -41,6 +41,7 @@ The AI provider path is complete for local fixture assessment workflows and incl
4141
`init`, `collect fixture`, `assess`, `map`, `generate --all`, `validate`, `export`.
4242
- [ ] GitHub, Okta, Auth0, Cloudflare, Akamai, OpenAI, and Anthropic fixture/live-collector test coverage pass, and supported fixture provider pipelines validate.
4343
- [ ] `openexit version` prints name, version, commit, and date from release build flags.
44+
- [ ] `openexit run --project <demo> --export --out <zip>` completes a collected project through assessment, mapping, generation, validation, status reporting, and bundle export.
4445
- [ ] `openexit status --project <demo>` reports inventory, assessment, mapping, generated artifacts, validation status, export readiness, and matching `--json` output.
4546
- [ ] `README.md`, `docs/cli.md`, `docs/security.md`, and this checklist reflect current behavior.
4647
- [ ] `examples/datadog-to-grafana/README.md` reproduces the primary local demo.

0 commit comments

Comments
 (0)