Skip to content

Automation/script to smoke test existing and new validation logic #1478

Open
@radeksimko

Description

@radeksimko

Context

After introducing enhanced validation in #1368 we have received a number of bug reports about false diagnostics:

and some more continue to trickle in:

Proposal

Instead of letting our users discover and report such cases of false diagnostics we can be more proactive and run our validation logic against publicly available repositories of Terraform code on GitHub, to perform a form of smoke testing of the validation logic.

This can in turn provide us with higher confidence in the validation logic and the schema, and detect at least some, if not most, of the bugs before they even reach users.

  • Identify some popular repositories with Terraform code on GitHub and the Terraform Registry
  • Create a script (Go code) which can run all existing validation logic
    • Context-less validation (SchemaModuleValidation, SchemaVariablesValidation)
    • Context-dependent validation (ReferenceValidation)
  • Run script against those repositories

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions