quartzctl is an open-source CLI tool designed to automate the installation and maintenance of multi-stage OpenTofu projects. It leverages a single YAML configuration to define stages, their dependencies, input variables, environment variables, and health checks, streamlining complex infrastructure deployments.
We were tasked with building a fully automated DevSecOps platform with Platform One and Ironbank as its core, while minimizing installation times and risk of transient failures. A high degree of flexibility was also needed, so as to facilitate rapid development of applications and infrastructure for hackathon style environments, all while keeping an eye on security for government customers. Initially developed as a loose conglomeration of bash scripts glued together by a makefile, this eventually became unmaintainable due to increasing complexity of requirements coupled with the expectation of quick turnaround with change requests.
Quartz is an open-source CLI tool designed to automate the full lifecycle of Kubernetes-based platform infrastructure with a focus on DevSecOps enablement. Originally inspired by U.S. government platform initiatives like PlatformOne and BigBang, Quartz simplifies the provisioning, configuration, and teardown of secure, GitOps-enabled environments. It leverages a top-level YAML configuration to drive installations, source secrets, and orchestrate tools like FluxCD or ArgoCD. With robust health checks, retry logic, and zero-interaction execution, Quartz empowers teams to build reproducible, production-like platforms in development or staging deployments.
addsubcommands to streamline adding new stagesgeneratesubcommands to create new projects with all required boilerplate (quartz.yaml, stages), options to enable/disable common items like eks, core helm charts, etc...- Plugin framework to expand beyond AWS and OpenTofu
- Unwind tightly coupled assumptions of the platform (ex: separate repositories vs monorepo, use of gitops, core application stack, etc...)
Delivery teams and capture teams should start with the quartzctl Capability Package. It includes the nine-document delivery set: capability summary, reference architecture, operating model, onboarding guide, runbook, troubleshooting guide, template package, value brief, and product page.
- Features
- Getting Started
- Usage
- Configuration
- Capability Documentation
- Development
- Security
- Contributing
- License
- Multi-Stage Management: Define and manage multiple OpenTofu stages with interdependencies.
- YAML Configuration: Centralized configuration file specifying stages, variables, and settings.
- Dependency Handling: Automatically determines the order of stage execution based on dependencies.
- Dynamic Variables: Pass output variables from one stage as input to another.
- Environment Management: Set environment variables and configuration values per stage.
- Health Checks: Execute pre- and post-apply/destroy health checks to ensure application stability.
- Resumable Installs: Resume from a named stage and skip completed stages only when drift checks show they are still in sync.
- State Repair: Import resources, release stale locks, inspect redacted state, and remove state entries through the CLI.
- Resilient Teardown: Destroy stages in reverse dependency order and preserve the backend until all stages cleanly destroy.
- Kubernetes Integration: Monitor Kubernetes deployment statuses and other conditions before proceeding.
- Extensibility: Support for additional features and integrations as needed.
You can install quartz using one of the following methods:
go install github.com/MetroStar/quartzctl@latestDownload the latest release from the Releases page and add it to your system's PATH.
quartz [command] [flags]check: Check environment, configuration and access for installer prerequisites. Usequartz check --ai-telemetryon a live Quartz cluster to verify Agent Gateway Prometheus scrape health, recent request volume, recent 2xx/5xx responses, and recent model-not-found/404 responses. Usequartz check --install-readinessafter install when you want a lightweight read on Flux convergence, app-delivery bootstrap readiness, and Ollama warmer status without re-reading the full cluster report.clean: Perform a full cleanup/teardown of the system. Pass--yes/-yto skip the confirmation prompt (useful for CI). Cleanup destroys stages in reverse dependency order, continues across stage failures, reports cumulative status, and preserves the state backend until all stages complete successfully.export: Export configured Kubernetes resources to yaml.info: Output configuration info for the current cluster.install: Perform a full install/update of the system. Pass--resume-from/-rto resume from a stage,--allow-deferralto enable OpenTofu deferred actions, and--yes/-yto skip confirmation prompts.login: Generate a kubeconfig for the current cluster (kubeconfigandrefresh-kubeconfigaliases).refresh-secrets: Trigger all external secrets to be refreshed immediately (rsalias).render: Write fully rendered configuration to yaml (--out/-o, default./out/quartz.generated.yaml).restart: Restart target Kubernetes resources. Defaults to deployments, daemonsets, and statefulsets; narrow with--kind/-k,--namespace/-n, and--name.tofu/tf: OpenTofu subcommands for configured stages.apply: Runtofu applyfor a stage (--stage <name>required).destroy: Runtofu destroyfor a stage (--stage <name>required).force-unlock: Runtofu force-unlockfor a stage (--stage <name>plus lock id).format/fmt: Runtofu fmtfor a stage (--stage <name>required).format-all: Runtofu fmtfor all stages.import: Runtofu importfor a stage (--stage <name>plus address/id).init: Runtofu initfor a stage (--stage <name>required).init-all: Runtofu initfor all stages.output: Runtofu outputfor a stage (--stage <name>required).plan: Runtofu planfor a stage (--stage <name>required).refresh: Runtofu refreshfor a stage (--stage <name>required).refresh-all: Runtofu refreshfor all stages.state: Inspect or modify stage state:list,show(sensitive values redacted), andrm/remove.validate: Runtofu validatefor a stage (--stage <name>required).version: Runtofu version.
help: Shows a list of commands or help for one command
--config: Path to the YAML configuration file (Optional, default:quartz.yaml).--secrets: Path to a YAML file containing secrets as an alternative to environment variables. For development use only (Optional).--help: Shows a list of commands or help for one command.--version: Print the version and build time.
SILENT: When set, all confirmation prompts are auto-confirmed (equivalent to passing--yes). Intended for non-interactive/CI use.
quartz --config=quartz.yaml installquartz install reports cluster success once the HelmRelease convergence gate is
green. On AI-enabled clusters, Ollama model pulling can continue in the
background after that point; the CLI now distinguishes between the durable
model store being ready and the later best-effort GPU warm-up pass. Use
quartz install --wait-for-models when you want the CLI itself to block until
the desired models are fully persisted.
When you just want the short follow-up signal after install, run:
quartz check --install-readinessThat check keeps the CLI thin while still surfacing the three things operators usually care about first: Flux HelmRelease convergence, app-delivery bootstrap, and AI model readiness.
The quartz.yaml file defines the stages and their configurations.
For Quartz platform environments, application delivery now treats these fields as first-class:
applications:
my-api:
repo_url: https://github.com/MetroStar/my-api.git
branch: main
path: deploy
settings:
post_deploy:
enabled: true
gate_environments: [stage, prod]
trigger_environments: [dev, stage, prod]path defaults to deploy. settings.post_deploy defaults to the policy shown above, which lets Quartz trigger app-owned verification hooks after deployment and use those results to gate stage and prod advancement without mutating Git. gitops.apps is now optional and deprecated for application delivery.
name: sampleenv # unique name of quartz cluster/environment
dns: # either of domain or zone must be specified
domain: "" # default <name>.<dns.zone>
zone: example.com # default parsed from dns.domain
aws:
region: us-east-1
The stage.yaml file allows for stage directories to override configuration from the cluster quartz.yaml or convention defaults.
# define input variables for the tofu stage and their source
# NOTE: all stages assume the existence of a `settings` input variable that receives the entire rendered config map unless overridden
vars:
# input variable <my_env_val> defined in variables.tf
my_env_val:
# populate with an environment variable
env: HOSTNAME
my_secret_val:
# populate with a value from the rendered secrets
secret: github.token
my_config_val:
# populate with a value from the rendered config
config: dns.domain
my_stage_output_val:
stage:
name: previous_stage
output: cluster.name
# health checks that determine if the dependent resources are available before or after performing an action on the stage
checks:
# group name, only shows up in logs
pre_install:
# when to run the checks in this group, before/after apply/destroy
before:
- apply
# define health checks derived from the state of a kubernetes resource
kubernetes:
- name: public-cert
namespace: cert-manager
kind: Certificate
state: Ready
timeout: 1200
- name: istio
kind: HelmRelease
state: Ready
init:
before:
- apply
# explicit ordering
order: 1
# check the quartz global configmap for a key/value, useful for confirming one time jobs were successful (Ex. initial admin password change, database setup)
state:
- key: "myapp.initialized"
value: "true"
api:
before:
- apply
order: 2
# perform http requests against the endpoint in a loop until success or timeout
http:
- path: /api/system/status
app: myapp
content:
json:
key: status
value: UP
# options for controlling what is or isn't destroyed (Ex. I'm tearing down the entire cluster, no reason to unconfigure Keycloak and waste time or risk it erroring)
# typically will only use either the include or exclude sections as the logic for using them both is messy and rarely useful
destroy:
include:
- "module.to_destroy"
exclude:
- "module.skip_destroy"
See the included samples for more details.
-
Clone the repository:
git clone https://github.com/MetroStar/quartzctl.git cd quartzctl -
Install mise:
curl https://mise.jdx.dev/install.sh | sh mise install -
Build the application:
mise run build
mise run testWe use golangci-lint for linting.
mise run lintIf you discover a security vulnerability, please follow the guidelines in our SECURITY.md file.
- Dependencies: We use Dependabot to keep dependencies up to date.
- CI/CD: All commits are tested via GitHub Actions workflows.
- Code Scanning: Static analysis is performed using CodeQL and other tools.
We welcome contributions! Please see our CONTRIBUTING.md for guidelines on how to get started.
This project is licensed under the Apache 2.0 License.
For questions or support, please open an issue or contact sblair@metrostar.com.
This project aims to comply with the OpenSSF Best Practices and has a Scorecard to reference.