Skip to content

CLI-boundary test coverage: query, diff, evidence publish, non-helmfile deployer #1806

Description

@njhensley

Split out of #1277 (DC4), which is closed — this is standalone CLI-coverage work, not part of the DC/dynamic-clusters epic (#1264).

Already covered (no work)

  • aicr evidence verify — UAT phase_verify on all 3 clouds, JSON result + exit asserted (tests/uat/aws/run:733).
  • real-cluster aicr snapshot — UAT phase_prep (tests/uat/aws/run:150) + CUJ chainsaw suites (functional; see gap fix: use workflow_run for PR coverage comments on fork PRs #5 for the result wrapper).
  • aicr validate — already emits CTRF JSON (dry-run.json, report.json), proven by assert-validate-*.yaml.

Gaps to close

  1. aicr query — zero coverage in tests/uat, tests/chainsaw, tests/e2e; only demos/query.md.
  2. aicr diff — completely unexercised anywhere. Highest gap.
  3. standalone aicr evidence publish — only the inline validate-time push runs in UAT (spec.validate.evidencetests/uat/aws/run:132-141); the standalone verb runs only in demos/evidence-demo.sh:256.
  4. A second bundle deployer beyond helmfile — every UAT config hardcodes deployer: helmfile (tests/uat/*/tests/h100-*-config.yaml); argocd/argocd-helm/flux are only structure-tested off-cluster in tests/chainsaw/cli/bundle-*. KWOK already deploys all 6 deployers (kwok/scripts/validate-scheduling.sh) — cheap place to get non-helmfile deploy coverage with a CTRF wrapper.
  5. snapshot structured-result wrapper — wrap the existing real-cluster snapshot outcome into a CTRF/JUnit record.

Each new invocation should emit a CTRF/JUnit-shaped result. No GPU-dependent required merge gate.

Metadata

Metadata

Assignees

Labels

Type

Fields

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions