Skip to content

Commit 150687a

Browse files
samtholiyaostermanautofix-ci[bot]aknysh
authored
implement schema validation for vendoring and CLI schemas (#1147)
* Update atmos schema init * updated schema * refactor schema name * fixed test case * processSchemas updated * fix golangci lint * fix the downloadSchemaFromURL lint * lint fix validate stacks * lint fix * fixed lints * fix lint * fix lint * fix lint * fixed the processCommandLineArgs lint * fix lint * fix lint * Update internal/exec/validate_stacks.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * Update pkg/config/utils.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * fix tests * fixed log messages * add new interface based downloader package * update the usage and remove old gogetter usage * [autofix.ci] apply automated fixes * add unit test cases * [autofix.ci] apply automated fixes * fix golangci lint * fix golangci-lint * fixed logging * fix vendor test case * [autofix.ci] apply automated fixes * added unit test cases to have more coverage * Update pkg/downloader/custom_github_detector.go Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> * fix test * [autofix.ci] apply automated fixes * golangci-lint fix * fix golangci-lint * ignore mock files in the test * fix rebase * fix golangci-lint * fixed logging * add validate schema initial code * temp commit * [autofix.ci] apply automated fixes * refactor the code * Fix test case * added unit testcase for validator * [autofix.ci] apply automated fixes * fixed golangci-lint * [autofix.ci] apply automated fixes * fix golangci-lint * used existing gogetter * unit test case updated * [autofix.ci] apply automated fixes * golangci-lint fix * [autofix.ci] apply automated fixes * more test cases * [autofix.ci] apply automated fixes * rebase fix * lint test * update validator * [autofix.ci] apply automated fixes * update validator * glob match updated * fix glob for mac and windows * add inline schema support * [autofix.ci] apply automated fixes * schema is the new name * codecov updated * fix test cases * updated atmos schemas library * fix macos test case * fix test cases * add more schemas * update logging structure * trying to ignore mock files * set flag and key * [autofix.ci] apply automated fixes * remove old unwanted test * fix merge issues * update file name * custom git detector * [autofix.ci] apply automated fixes * added global schema * fix golangci lint * updated parsing logic * fix test case * fix test case * reduce complexity * updated describe_config test * fix snapshots * fix test case * updated snapshot * fix tests * Update cmd/validate_schema.go Co-authored-by: Andriy Knysh <[email protected]> * Update internal/exec/validate_schema.go Co-authored-by: Andriy Knysh <[email protected]> * fix comments * [autofix.ci] apply automated fixes * added negative test cases * [autofix.ci] apply automated fixes * fix suggestions * fix golangci-lint * updated the help * udpate tests * fix golangci-lint * add help examples for validate schema * add website docs * remove unwanted flag * updated the doc * fix coderabbit * updated schema --------- Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Andriy Knysh <[email protected]>
1 parent f42f14b commit 150687a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+4533
-13
lines changed

‎.golangci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ linters-settings:
115115
- name: add-constant
116116
arguments:
117117
- maxLitCount: "3"
118-
allowStrs: '"","image","error","path","import","path","%w","%s"'
118+
allowStrs: '"","image","error","path","import","path","%w","%s","file","/"'
119119
allowInts: "0,1,2,3,4"
120120
allowFloats: "0.0,0.,1.0,1.,2.0,2."
121121
- name: argument-limit

‎Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,8 @@ testacc: get
6161

6262
testacc-cover: get
6363
@echo "Running tests with coverage"
64-
go test $(TEST) -v $(TESTARGS) -timeout 20m -coverprofile=coverage.out
64+
go test $(TEST) -v $(TESTARGS) -timeout 20m -coverprofile=coverage.out.tmp
65+
cat coverage.out.tmp | grep -v "mock_" > coverage.out
6566

6667
# Run acceptance tests with coverage report
6768
testacc-coverage: testacc-cover

‎cmd/markdown/atmos_validate_schema_usage.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
- Validate all the schemas
2+
3+
```bash
4+
$ atmos validate schema
5+
```
6+
7+
- Validate specific schema
8+
9+
```
10+
$ atmos validate schema [schemaToValidate]
11+
```

‎cmd/validate_schema.go

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
package cmd
2+
3+
import (
4+
"errors"
5+
6+
log "github.com/charmbracelet/log"
7+
"github.com/cloudposse/atmos/internal/exec"
8+
u "github.com/cloudposse/atmos/pkg/utils"
9+
"github.com/spf13/cobra"
10+
)
11+
12+
// ValidateSchemaCmd represents the 'atmos validate schema' command.
13+
//
14+
// This command reads the 'schemas' section from the atmos.yaml configuration file,
15+
// where each schema entry specifies a JSON schema path and a glob pattern for matching YAML files.
16+
//
17+
// For each entry:
18+
// - The JSON schema is loaded.
19+
// - All YAML files matching the glob pattern are discovered.
20+
// - Each YAML file is converted to JSON and validated against the schema.
21+
//
22+
// This command ensures that configuration files conform to expected structures and helps
23+
// catch errors early in the development or deployment process.
24+
var ValidateSchemaCmd = &cobra.Command{
25+
Use: "schema",
26+
Short: "Validate YAML files against JSON schemas defined in atmos.yaml",
27+
Long: `The validate schema command reads the ` + "`" + `schemas` + "`" + ` section of the atmos.yaml file
28+
and validates matching YAML files against their corresponding JSON schemas.
29+
30+
Each entry under ` + "`" + `schemas` + "`" + ` should define:
31+
- ` + "`" + `schema` + "`" + `: The path to the JSON schema file.
32+
- ` + "`" + `matches` + "`" + `: A glob pattern that specifies which YAML files to validate.
33+
34+
For every schema entry:
35+
- The JSON schema is loaded from the specified path.
36+
- All files matching the glob pattern are collected.
37+
- Each matching YAML file is parsed and converted to JSON.
38+
- The converted YAML is validated against the schema.
39+
40+
This command helps ensure that configuration files follow a defined structure
41+
and are compliant with expected formats, reducing configuration drift and runtime errors.
42+
`,
43+
FParseErrWhitelist: struct{ UnknownFlags bool }{UnknownFlags: false},
44+
Args: cobra.MaximumNArgs(1),
45+
Run: func(cmd *cobra.Command, args []string) {
46+
// Check Atmos configuration
47+
checkAtmosConfig()
48+
schema := ""
49+
key := ""
50+
if len(args) > 0 {
51+
key = args[0] // Use provided argument
52+
}
53+
54+
if cmd.Flags().Changed("schemas-atmos-manifest") {
55+
schema, _ = cmd.Flags().GetString("schemas-atmos-manifest")
56+
}
57+
58+
if key == "" && schema != "" {
59+
log.Error("key not provided for the schema to be used")
60+
u.OsExit(1)
61+
}
62+
63+
if err := exec.NewAtmosValidatorExecutor(&atmosConfig).ExecuteAtmosValidateSchemaCmd(key, schema); err != nil {
64+
if errors.Is(err, exec.ErrInvalidYAML) {
65+
u.OsExit(1)
66+
}
67+
u.PrintErrorMarkdownAndExit("", err, "")
68+
}
69+
},
70+
}
71+
72+
func init() {
73+
ValidateSchemaCmd.PersistentFlags().String("schemas-atmos-manifest", "", "Specifies the path to a JSON schema file used to validate the structure and content of the Atmos manifest file")
74+
validateCmd.AddCommand(ValidateSchemaCmd)
75+
}

‎codecov.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ coverage:
99
threshold: 2% # Allow a small drop in coverage
1010
base: auto
1111
ignore:
12+
- "**/mock_*.go" # Adjust this pattern based on your project structure
1213
- "mock_*.go" # Adjust this pattern based on your project structure
1314

1415
comment:

‎examples/demo-schemas/atmos.yaml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
schemas:
2+
schemaFromFile:
3+
manifest: ./manifest.json
4+
matches:
5+
- config.yaml
6+
schemaFromInternet:
7+
manifest: https://json.schemastore.org/bower.json
8+
matches:
9+
- bower.yaml
10+
schemaFromInline:
11+
manifest: |
12+
{
13+
"type": "object",
14+
"properties": {
15+
"name": { "type": "string" },
16+
"age": { "type": "integer" }
17+
},
18+
"required": ["name"]
19+
}
20+
matches:
21+
- inline.yaml

‎examples/demo-schemas/bower.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
name: "my-project"
2+
version: "1.0.0"
3+
description: "A sample project to demonstrate bower.json schema validation."
4+
main: "index.js"
5+
authors:
6+
- "Jane Doe <[email protected]>"
7+
license: "MIT"
8+
dependencies:
9+
jquery: "^3.5.1"
10+
lodash: "^4.17.20"

‎examples/demo-schemas/config.yaml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
name: "MyApp"
2+
version: "1.0.0"
3+
enabled: true
4+
settings:
5+
timeout: 30
6+
debug: false
7+
users:
8+
- id: 1
9+
name: "Alice"
10+
role: "admin"
11+
- id: 2
12+
name: "Bob"
13+
role: "user"
14+
- id: 3
15+
name: "Charlie"
16+
role: "guest"

‎examples/demo-schemas/inline.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
name: John
2+
age: 30

‎examples/demo-schemas/manifest.json

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"type": "object",
4+
"properties": {
5+
"name": {
6+
"type": "string"
7+
},
8+
"version": {
9+
"type": "string",
10+
"pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$"
11+
},
12+
"enabled": {
13+
"type": "boolean"
14+
},
15+
"settings": {
16+
"type": "object",
17+
"properties": {
18+
"timeout": {
19+
"type": "integer",
20+
"minimum": 1
21+
},
22+
"debug": {
23+
"type": "boolean"
24+
}
25+
},
26+
"required": ["timeout"]
27+
},
28+
"users": {
29+
"type": "array",
30+
"items": {
31+
"type": "object",
32+
"properties": {
33+
"id": {
34+
"type": "integer"
35+
},
36+
"name": {
37+
"type": "string"
38+
},
39+
"role": {
40+
"type": "string",
41+
"enum": ["admin", "user", "guest"]
42+
}
43+
},
44+
"required": ["id", "name", "role"]
45+
}
46+
}
47+
},
48+
"required": ["name", "version", "enabled", "settings"]
49+
}
50+

‎go.mod

Lines changed: 3 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎go.sum

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎internal/exec/validate_schema.go

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
package exec
2+
3+
import (
4+
"fmt"
5+
6+
log "github.com/charmbracelet/log"
7+
8+
"github.com/cloudposse/atmos/pkg/downloader"
9+
"github.com/cloudposse/atmos/pkg/filematch"
10+
"github.com/cloudposse/atmos/pkg/schema"
11+
"github.com/cloudposse/atmos/pkg/validator"
12+
)
13+
14+
var ErrInvalidYAML = fmt.Errorf("invalid YAML")
15+
16+
type ErrInvalidPattern struct {
17+
Pattern string
18+
err error
19+
}
20+
21+
func (e ErrInvalidPattern) Error() string {
22+
return fmt.Sprintf("invalid pattern %q: %v", e.Pattern, e.err)
23+
}
24+
25+
type atmosValidatorExecutor struct {
26+
validator validator.Validator
27+
fileDownloader downloader.FileDownloader
28+
fileMatcher filematch.FileMatcher
29+
atmosConfig *schema.AtmosConfiguration
30+
}
31+
32+
func NewAtmosValidatorExecutor(atmosConfig *schema.AtmosConfiguration) *atmosValidatorExecutor {
33+
fileDownloader := downloader.NewGoGetterDownloader(atmosConfig)
34+
return &atmosValidatorExecutor{
35+
validator: validator.NewYAMLSchemaValidator(atmosConfig),
36+
fileDownloader: fileDownloader,
37+
fileMatcher: filematch.NewGlobMatcher(),
38+
atmosConfig: atmosConfig,
39+
}
40+
}
41+
42+
func (av *atmosValidatorExecutor) ExecuteAtmosValidateSchemaCmd(sourceKey string, customSchema string) error {
43+
validationSchemaWithFiles, err := av.buildValidationSchema(sourceKey, customSchema)
44+
if err != nil {
45+
return err
46+
}
47+
48+
totalErrCount, err := av.validateSchemas(validationSchemaWithFiles)
49+
if err != nil {
50+
return err
51+
}
52+
53+
if totalErrCount > 0 {
54+
return ErrInvalidYAML
55+
}
56+
return nil
57+
}
58+
59+
func (av *atmosValidatorExecutor) buildValidationSchema(sourceKey, customSchema string) (map[string][]string, error) {
60+
validationSchemaWithFiles := make(map[string][]string)
61+
log.Debug("Building validation schema with files", "sourceKey", sourceKey, "customSchema", customSchema, "schemas", av.atmosConfig.Schemas)
62+
for k := range av.atmosConfig.Schemas {
63+
if av.shouldSkipSchema(k, sourceKey) {
64+
log.Debug("Skipping schema", "key", k, "sourceKey", sourceKey)
65+
continue
66+
}
67+
68+
schemaValue := av.prepareSchemaValue(k, sourceKey, customSchema)
69+
if schemaValue.Schema == "" {
70+
log.Debug("Skipping schema with empty schema", "key", k, "sourceKey", sourceKey, "schemaValue", schemaValue)
71+
continue
72+
}
73+
74+
files, err := av.fileMatcher.MatchFiles(schemaValue.Matches)
75+
if err != nil {
76+
return nil, err
77+
}
78+
log.Debug("Files matched", "schema", schemaValue.Schema, "matcher", schemaValue.Matches, "filesMatched", files)
79+
validationSchemaWithFiles[schemaValue.Schema] = files
80+
}
81+
log.Debug("Validation schema with files", "validationSchemaWithFiles", validationSchemaWithFiles)
82+
return validationSchemaWithFiles, nil
83+
}
84+
85+
func (av *atmosValidatorExecutor) shouldSkipSchema(k, sourceKey string) bool {
86+
return (sourceKey != "" && sourceKey != k) || k == "cue" || k == "opa" || k == "jsonschema"
87+
}
88+
89+
func (av *atmosValidatorExecutor) prepareSchemaValue(k, sourceKey, customSchema string) schema.SchemaRegistry {
90+
value := av.atmosConfig.GetSchemaRegistry(k)
91+
if sourceKey != "" && customSchema != "" {
92+
value.Schema = customSchema
93+
}
94+
switch {
95+
case value.Schema == "" && value.Manifest == "":
96+
value.Schema = fmt.Sprintf("atmos://schema/%s/manifest/1.0", k)
97+
case value.Schema == "" && value.Manifest != "":
98+
value.Schema = value.Manifest
99+
case customSchema != "":
100+
value.Schema = customSchema
101+
}
102+
if len(value.Matches) == 0 && sourceKey == "atmos" {
103+
value.Matches = []string{"atmos.yaml", "atmos.yml"}
104+
}
105+
106+
return value
107+
}
108+
109+
func (av *atmosValidatorExecutor) validateSchemas(schemas map[string][]string) (uint, error) {
110+
totalErrCount := uint(0)
111+
for k, files := range schemas {
112+
errCount, err := av.printValidation(k, files)
113+
if err != nil {
114+
return 0, err
115+
}
116+
totalErrCount += errCount
117+
}
118+
return totalErrCount, nil
119+
}
120+
121+
func (av *atmosValidatorExecutor) printValidation(schema string, files []string) (uint, error) {
122+
count := uint(0)
123+
for _, file := range files {
124+
log.Debug("validating", "schema", schema, "file", file)
125+
validationErrors, err := av.validator.ValidateYAMLSchema(schema, file)
126+
if err != nil {
127+
return count, err
128+
}
129+
if len(validationErrors) == 0 {
130+
log.Info("No Validation Errors", "file", file, "schema", schema)
131+
continue
132+
}
133+
log.Error("Invalid YAML", "file", file)
134+
for _, err := range validationErrors {
135+
log.Error("", "file", file, "field", err.Field(), "type", err.Type(), "description", err.Description())
136+
count++
137+
}
138+
}
139+
return count, nil
140+
}

0 commit comments

Comments
 (0)