Skip to content

Add CLI smoke checks against repo-root test_data/ fixtures in CI #61

@hbelmiro

Description

@hbelmiro

Summary

We want CI to run selected scripts/ CLIs against the committed repo-root test_data/ fixtures (the “golden snapshot”) and require exit code 0. This provides an integration-style sanity check that scripts work end-to-end on representative assets.

Proposed CI location

Add steps to .github/workflows/scripts-tests.yml (for the scripts matrix entry) so these checks run when scripts change.

Candidate CLIs

These CLIs are intended to be “read-only” checks (no file modifications):

  • scripts/check_base_image_tags/check_base_image_tags.py
  • scripts/validate_components/validate_components.py
  • scripts/validate_base_images/validate_base_images.py
  • scripts/validate_metadata/validate_metadata.py
  • scripts/generate_readme (check mode)

Also, evaluate new CLIs that were included after the creation of this issue.

Constraints / design notes

  • The CLIs should be runnable against test_data/ without requiring mutation of fixtures or the repository.
  • If a CLI currently can’t target test_data/ due to hard-coded assumptions (e.g., requiring components/ at repo root, assuming importable packages), options include:
    • removing those repo-layout assumptions and operating from file paths; or
    • introducing a flag like --repo-root PATH to treat test_data/ as the logical root.

Acceptance criteria

  • A set of steps in .github/workflows/scripts-tests.yml runs the chosen CLIs against test_data/ and fails the workflow on non-zero exit.
  • The steps are gated so they don’t run in the .github/scripts matrix entry.
  • The smoke checks are stable across the Python versions used in CI.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions