Skip to content

README.md

Latest commit

 

History

History
234 lines (180 loc) · 10.6 KB

README.md

File metadata and controls

234 lines (180 loc) · 10.6 KB

flux-schema

release test cve-scan license slsa

Flux CLI plugin for Kubernetes schema extraction and manifests validation.

Note

This repository is in early development and the plugin system is not yet available in a stable release of Flux. The instructions for installing and using the plugin will be added here when RFC-0013 has been implemented and released in Flux 2.9 or later.

Available Commands

  • flux-schema validate [paths...]: Validate Kubernetes manifests against JSON Schemas
    • --schema-location: URL or file path for schemas (repeatable, tried in order); default points at the built-in catalog
    • --skip-missing-schemas: Skip documents for which no schema can be found
    • --skip-kind: Skip documents matching Kind or apiVersion/Kind (repeatable)
    • --fail-fast: Exit after the first invalid document
    • --concurrent: Number of concurrent workers (default 8)
    • --insecure-skip-tls-verify: Disable TLS certificate verification when fetching schemas over HTTPS
    • -v, --verbose: Print a line for every document, including valid and skipped ones
    • --config: Path to a YAML file supplying default values for validate flags (env: FLUX_SCHEMA_CONFIG)
  • flux-schema extract crd [files...]: Extract JSON Schema from Kubernetes CRD YAMLs
    • -d, --output-dir: Directory to write JSON Schema files to (mutually exclusive with --output-archive)
    • -a, --output-archive: Path to write a gzipped tar archive of JSON Schema files to
    • -f, --output-format: Go template for output file paths (default: {{ .Group }}/{{ .Kind }}_{{ .Version }}.json)
    • --strip-description: Drop description fields from the generated schemas to reduce their size
  • flux-schema extract k8s [swagger-file]: Extract JSON Schema from a Kubernetes OpenAPI v2 swagger document
    • --version X.Y.Z: Fetch the swagger from kubernetes/kubernetes for the given release tag (mutually exclusive with a swagger file)
    • -d, --output-dir, -f, --output-format, --strip-description: same as extract crd

Kubernetes Manifests Validation

The validate command reads Kubernetes YAML manifests from one or more files or directories and validates each document against a JSON Schema resolved from its apiVersion and kind.

When no --schema-location is given, validate uses the flux-schema catalog, which covers the latest Kubernetes APIs, the stable channel of Gateway API, and the Flux ecosystem CRDs:

flux-schema validate ./manifests

To validate against your own schemas generated by flux-schema extract, pass the directory as --schema-location. Bare paths and URLs are auto-expanded to the catalog layout {{.Group}}/{{.Kind}}_{{.Version}}.json:

flux-schema validate ./manifests --schema-location ./my-schemas

For a different layout, pass a full Go template ending in .json:

flux-schema validate ./manifests \
  --skip-missing-schemas \
  --schema-location './schemas/{{.Kind}}-{{.GroupPrefix}}-{{.Version}}.json'

Template variables are .Group, .GroupPrefix, .Kind, and .Version.

The --schema-location flag is repeatable and locations are tried in order (the first match wins). Pass the literal value default to include the flux-schema catalog alongside your own schemas:

flux-schema validate ./manifests \
  --schema-location default \
  --schema-location https://raw.githubusercontent.com/datreeio/CRDs-catalog/main \
  --schema-location './schemas/{{.Kind}}-{{.GroupPrefix}}-{{.Version}}.json'

Manifests can also be piped in and certain documents skipped with --skip-kind:

kustomize build . | flux-schema validate \
  --skip-kind 'v1/Secret' \
  --skip-kind 'source.toolkit.fluxcd.io/v1/ExternalArtifact'

Output example with validation errors:

manifests/sources.yaml - Bucket/apps/s3-data is invalid: schema validation failed
  - /spec: missing property 'bucketName'
  - /spec/interval: got number, want string
  - /spec/secretRef/name: got object, want string
  - /spec: additional properties 'force' not allowed
manifests/sources.yaml - OCIRepository/apps/podinfo is invalid: YAML parse failed
  - line 18: key "app.kubernetes.io/name" already set in map
manifests/sources.yaml - HelmChart/apps/redis is valid
manifests/sources.yaml - Secret/apps/auth-sops is skipped: kind skipped
Summary: 4 resources found in 1 file - Valid: 1, Invalid: 2, Skipped: 1

A non-zero exit code is returned when any document is invalid or errored.

Validation rules

  • YAML documents with duplicate keys are rejected matching Flux behavior.
  • Documents missing both metadata.name and metadata.generateName are flagged as invalid matching Kubernetes API behavior.
  • Schemas produced by flux-schema extract crd close objects with additionalProperties: false, so undocumented fields under spec fail validation.
  • String formats duration, date, datetime/date-time, and time are validated matching Kubernetes API conventions.

Config file

Flag values can be pre-set in a YAML config file and referenced with --config or with the FLUX_SCHEMA_CONFIG environment variable. This keeps long invocations out of CI scripts and makes validation reproducible across developers:

flux-schema validate ./manifests --config .flux-schema.yaml

The file has a version (required, must be "1") and a validate section whose keys mirror the CLI flag names:

version: "1"
validate:
  schema-location:
    - default
    - https://raw.githubusercontent.com/datreeio/CRDs-catalog/main
  skip-kind:
    - v1/Secret
    - source.toolkit.fluxcd.io/v1/ExternalArtifact
  skip-missing-schemas: false
  verbose: true
  fail-fast: false
  concurrent: 8
  insecure-skip-tls-verify: false

Rules:

  • CLI flags override config values. Setting --verbose=false wins over verbose: true in the file.
  • Setting --config overrides FLUX_SCHEMA_CONFIG. When both are set, the flag wins and the env var is ignored.
  • Manifest paths stay positional. The config file configures how to validate; paths are given on the command line.

Kubernetes CRD Extraction

The extract crd command reads Kubernetes CustomResourceDefinition YAML and writes one JSON Schema file per CRD version. The input can be a bare CRD, a List of CRDs, or a multi-document YAML stream.

Generate schemas for every CRD installed in a cluster, using the per-group-directory layout:

kubectl get crds -o yaml | flux-schema extract crd -d ./schemas

You can supply -f, --output-format with a Go template to change the layout, e.g. the kubeconform/kubeval flat layout:

flux-schema extract crd crds.yaml -f '{{ .Kind }}-{{ .GroupPrefix }}-{{ .Version }}.json'

The output is compatible with kubeconform and kubeval, making this command a drop-in replacement for kubeconform's openapi2jsonschema.py script.

To bundle the schemas into a gzipped tar archive instead of writing to a directory, use -a, --output-archive:

kustomize build config/crd | flux-schema extract crd -a dist/crd-schemas.tar.gz

The archive path must end in .tar.gz or .tgz, and its parent directory is created if missing. -f, --output-format still controls the entry paths inside the archive, so a nested template produces subdirectories within the tarball.

Supported template variables (all lowercased at render time):

Variable Example
.Group source.toolkit.fluxcd.io
.GroupPrefix source
.Kind gitrepository
.Version v1

An empty .Group (Kubernetes core API, e.g. apiVersion: v1) is normalized to core, and .GroupPrefix is derived from .Group when unset. So a core Pod renders as core/pod_v1.json with the default template and resolves to the same path when validate looks up its schema.

Note that the generated schemas apply the following OpenAPI → JSON Schema transformations:

  • Objects with properties are closed with additionalProperties: false, except under nodes marked with x-kubernetes-preserve-unknown-fields: true, which stay open so free-form maps validate correctly.
  • Integer-or-string fields are rewritten to oneOf: [{type: string}, {type: integer}]. Both the legacy format: int-or-string and the structural x-kubernetes-int-or-string: true forms are recognized.

Kubernetes OpenAPI Extraction

The extract k8s command reads a Kubernetes OpenAPI v2 swagger document and writes one JSON Schema file per kind listed under x-kubernetes-group-version-kind. Helper types (e.g. PodSpec, ObjectMeta) are not emitted as standalone files — they are inlined into the kinds that reference them.

Fetch the upstream swagger for a Kubernetes release and generate schemas:

flux-schema extract k8s --version 1.35.0 -d ./schemas

Supply the swagger file from the cluster:

kubectl get --raw /openapi/v2 | flux-schema extract k8s -d ./schemas

The same -f, --output-format template used by extract crd works here, so the generated files remain resolvable by validate with a single --schema-location for both built-ins and CRDs. Core API kinds (apiVersion: v1) render under the core/ group directory.

The emitted schemas are the standalone-strict variant:

  • Every $ref is inlined so schemas have no cross-file dependencies.
  • Objects with properties are closed with additionalProperties: false, except under nodes marked x-kubernetes-preserve-unknown-fields: true.
  • Integer-or-string fields are rewritten to oneOf: [{type: string}, {type: integer}].
  • Optional fields are marked nullable (type: [<t>, "null"]), matching the Kubernetes API server's behavior of accepting null for unset optional values.
  • apiVersion and kind are injected into every kind's properties and required list.