Skip to content

Commit 349b4f9

Browse files
committed
Add AGENTS.md
1 parent 1cfa6ef commit 349b4f9

1 file changed

Lines changed: 57 additions & 0 deletions

File tree

AGENTS.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Kubestack AGENTS.md file
2+
3+
Kubestack is an OpenTofu/Terraform framework for platform engineering teams building Kubernetes-based platforms.
4+
It provides re-usable modules to manage clusters, node-pools and platform services from a unified configuration using a GitOps workflow.
5+
6+
* cluster modules provision managed Kubernetes offerings from cloud providers and all their respective infrastructure requirements
7+
* node-pool modules are used to configure the node pools attached to the respective cluster
8+
* platform service modules install services that need to exist on clusters before applications can be deployed
9+
10+
Inheritance-based configuration to avoid drift between environments, and keeping infrastructure and application environments separate to avoid blockers between infrastructure and application changes are cornerstones of Kubestack's design.
11+
12+
**This file documents contribution guidelines for the Kubestack framework itself, not for users of the framework.**
13+
14+
## Repository and module layout
15+
16+
* the repository root has directories for `common/` modules shared between all providers, and provider-specific module directories
17+
* provider-specific module directory names follow the OpenTofu/Terraform provider name for that cloud, e.g. `aws`, `azurerm` or `google`
18+
* in each provider-specific module directory there is a `cluster/` directory holding the cluster module and its node-pool submodule in the `cluster/node-pool` directory
19+
* the configuration and metadata module calls, as well as the main resource for the respective module (cluster resource for cluster modules, node-pool resource for node-pool modules) must always be in the file `main.tf`
20+
* every cluster provisioned by Kubestack must have a default node pool; the default node pool must be provisioned by calling the cluster's node-pool submodule in a `node_pool.tf` file, with the following exceptions:
21+
* if the cloud provider mandates a built-in default node pool that cannot be disabled (e.g. AKS), use the provider's built-in default node pool instead; `node_pool.tf` is not required in this case
22+
* if the cloud provider includes a default node pool that can be disabled (e.g. GKE), it must be disabled and the node-pool submodule must still be used in `node_pool.tf`
23+
* kubeconfig and default ingress related code must be in `kubeconfig.tf` and `ingress.tf` for all cluster modules
24+
* outputs must be defined in `outputs.tf`
25+
* moved blocks must be in `moved.tf`
26+
* variables must be defined in `variables.tf`
27+
* a data source used by only one resource must be placed directly in front of that resource; if a data source is used by multiple resources or supports conditional logic, it must be placed in a separate `data_sources.tf` file
28+
* additional files like `vpc.tf`, `roles.tf`, `launch_template.tf` etc. may be created if grouping resources into separate files improves maintainability
29+
30+
### Variables and Outputs
31+
* all Kubestack modules share a common set of variables and outputs per module type
32+
* configuration inheritance variables:
33+
* every module written for the Kubestack framework must define both a `configuration` variable of type `map(object)` and a `configuration_base_key` variable of type `string`; the `configuration_base_key` variable defaults to `apps` so it is optional for users of the module, but both variables must always be defined when writing framework modules
34+
* the map key of `configuration` is the environment name and the object defines the module and provider-specific configuration attributes
35+
* all configuration object attributes must be marked optional and must not define a default, because the inheritance implementation relies on unset attributes being null to determine whether to inherit or overwrite an attribute for the current environment
36+
* inside the module the configuration module must be called, passing in `var.configuration` and `var.configuration_base_key`; the resulting merged config for the current environment (`terraform.workspace`) must be made available as `local.cfg`
37+
* `local.cfg.<key>` must be used as the input to any resource that requires a value from the module's configuration
38+
* Kubestack prefers upstream defaults by passing `null` to resource arguments
39+
* if a default must be set, use the `try(coalesce(<key>, null), <default>)` pattern; this ensures that any nested key error or a null value from coalesce is caught by try, so the default only has to be specified once
40+
* cluster module outputs:
41+
* `cluster`: all attributes the cluster resource returns
42+
* `current_config`: the merged output of the configuration module (`local.cfg`)
43+
* `current_metadata`: the entire output of the metadata module
44+
* `kubeconfig`: the yaml encoded kubeconfig for the provisioned cluster; the output is used to configure the provider for platform service modules to be installed on the cluster
45+
* do not add additional outputs unless there is no alternative; exceptions must not be taken lightly to avoid module user experience drifting apart between providers
46+
* node-pool module variables and outputs:
47+
* in addition to the common `configuration` and `configuration_base_key` variables, node pools also require `cluster` and `cluster_metadata` variables that expect the data from the cluster's corresponding `cluster` and `current_metadata` outputs
48+
* do not add additional variables or outputs unless there is no alternative; the `aws/cluster` and `aws/cluster/node-pool` modules are an example of a case where additional variables were necessary
49+
50+
## Testing instructions
51+
* testing the Kubestack framework is expensive because real cloud infrastructure has to be created and cleanly deprovisioned as part of the integration tests
52+
* the multi-cloud platform configuration for the tests can be found in the `tests/` directory
53+
* a Makefile provides common development targets to run locally and ensures they are run in a container with the correct dependencies and correct working directory
54+
* never run OpenTofu/Terraform commands manually outside one of the specified make targets
55+
* `make validate` is safe to run at any time and must be run to validate any changes
56+
* never run `make test` or `make cleanup` unless explicitly asked to by the user
57+
* the `common/configuration` and `common/metadata` modules have unit tests; after making any changes to these modules, run `tofu test` inside the respective module directory

0 commit comments

Comments
 (0)