You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/standards/kubernetes_standards.md
+22-18Lines changed: 22 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,33 +4,29 @@
4
4
### Use a managed Kubernetes service
5
5
Managed Kubernetes services such as Azure Kubernetes Service (AKS) in Azure or Elastic Kubernetes Service (EKS) in AWS are used as opposed to any IaaS Kubernetes implementation.
6
6
7
-
This is because managed Kubernetes services abstract the maintenence and configuration of master nodes to the cloud provider, meaning teams only need to support the worker nodes where services run. Maintaining a full Kubernetes cluster can be very complicated and requires a high level of in depth Kubernetes and networking knowledge which can be a barrier to entry for some teams. Using a managed service significantly reduces this complexity.
7
+
This is because managed Kubernetes services abstract the maintenance and configuration of master nodes to the cloud provider, meaning teams only need to support the worker nodes where services run. Maintaining a full Kubernetes cluster can be very complicated and requires a high level of in depth Kubernetes and networking knowledge which can be a barrier to entry for some teams. Using a managed service significantly reduces this complexity.
8
8
9
9
### Use Helm for packaging deployments
10
10
[Helm](https://helm.sh/) is a tool to bundle individual Kubernetes configuration files into single deployable packages. This significantly reduces the complexity of configuring a cluster and deploying applications to it.
11
11
12
-
Helm 3 must be used and teams should never use Helm 2 due to security risk introduced by running Tiller in a cluster.
12
+
Helm >=3 must be used and teams should never use Helm 2 due to security risk introduced by running Tiller in a cluster.
13
13
14
-
Teams are also encouraged to use a Helm Library chart to reduce duplication of Helm charts across multiple services. Such as [this one](https://github.com/DEFRA/ffc-helm-library) used by the Future Farming and Countryside Programme (FFC).
14
+
Teams are also encouraged to use a Helm Library chart to reduce duplication of Helm charts across multiple services. Such as [this one](https://github.com/DEFRA/ffc-helm-library) used by the Farming and Countryside Programme (FCP).
15
15
16
16
### Use ConfigMaps for configuration
17
17
Configuration for an application running in a pod should be passed to the pod via a `ConfigMap` Kubernetes resource.
18
18
The `ConfigMap` is more flexible than just using environment variables alone, and as well as supporting file based values, allows decoupling of pod definitions from configuration definitions.
19
19
20
-
Environment specific values should be overriden during the Helm deployment.
20
+
Environment specific values should be overridden during the Helm deployment.
21
21
22
22
Sensitive values should never be passed to a `ConfigMap`.
23
23
24
24
### Secrets
25
-
Where possible, secrets should not be stored within an application or Kubernetes pod. Instead, when communicating with supported cloud infrastructure, clusters should use [AAD Pod Identity](https://github.com/Azure/aad-pod-identity) in Azure or [IAM role for Service Accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) in AWS.
25
+
Where possible, secrets should not be stored within an application or Kubernetes pod. Instead, when communicating with supported cloud infrastructure, clusters should use [Entra ID Workload Identities](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet) in Azure or [IAM role for Service Accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) in AWS.
26
26
27
27
When secrets in a pod are unavoidable, for example when a third party API key is needed, secrets should be injected into pods during deployment.
28
28
29
-
**Note** the precise mechanism for managing this is still being reviewed with in a collaboration between Cloud Technology Working Group (CTWG), Future Farming and Countryside Programme (FFC) and Europe and Trade Delivery Portfolio (EuTDP).
30
-
31
-
This document will be updated when an agreement is reached.
32
-
33
-
The Kubernetes `Secrets` resource type must not be used as data is only Base64 encrypted.
29
+
> Note: the Kubernetes `Secrets` resource type stores data as Base64 encoded as opposed to encryption. Ensure appropriate access controls are in place for reading cluster secrets.
34
30
35
31
### Labels
36
32
Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system.
@@ -42,7 +38,7 @@ In order to take full advantage of using labels, they should be applied on every
42
38
#### Required labels
43
39
Each Helm chart templated resource should have the below labels. Example placeholders are provided for values.
44
40
45
-
```
41
+
```yaml
46
42
metadata:
47
43
labels:
48
44
app: {{ quote .Values.namespace }}
@@ -60,7 +56,8 @@ metadata:
60
56
61
57
### Selectors
62
58
Services selectors should be matched by app and name. Selectors should be consistent otherwise updates to Helm charts will be rejected.
63
-
```
59
+
60
+
```yaml
64
61
selector:
65
62
app: {{ quote .Values.name }}
66
63
app.kubernetes.io/name: {{ quote .Values.name }}
@@ -83,7 +80,7 @@ Clusters will limit available resources within a namespace using a `resourceQuot
83
80
84
81
An example `ResourceQuota` definition is included below
85
82
86
-
```
83
+
```yaml
87
84
apiVersion: v1
88
85
kind: ResourceQuota
89
86
metadata:
@@ -99,9 +96,9 @@ spec:
99
96
### Pod priority
100
97
Kubernetes Pods can have priority levels. Priority indicates the importance of a pod relative to other pods. If a pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority pods to make scheduling of the pending pod possible.
101
98
102
-
In the event of over utilisation of a cluster, Kubernetes will start to kill lower priorty pods first to maintain stability.
99
+
In the event of over utilisation of a cluster, Kubernetes will start to kill lower priority pods first to maintain stability.
103
100
104
-
Clusters should include pod priority classes that teams can consume based on their service needs. The below gives examples of the Pod Priorty classes available in FFC clusters.
101
+
Clusters should include pod priority classes that teams can consume based on their service needs. The below gives examples of the Pod Priority classes available in FFC clusters.
105
102
106
103
#### High (1000)
107
104
Reserved primarily for customer facing or critical workload pods.
@@ -114,7 +111,7 @@ For pods where downtime is more tolerable.
114
111
115
112
Below is a full example of a `priorityClass` resource definition for reference.
116
113
117
-
```
114
+
```yaml
118
115
apiVersion: scheduling.k8s.io/v1
119
116
kind: PriorityClass
120
117
metadata:
@@ -128,6 +125,13 @@ description: "This priority class should be used for most services"
128
125
In the event a cluster has to make a choice between killing one of two services sharing the same priority level, the resource profile configuration will influence which is killed.
129
126
130
127
### Probes
131
-
To increase the stability and predicatability of a Kubernetes cluster, services should make use of both readiness and liveness probes unless there is a significant reason not to.
128
+
To increase the stability and predictability of a Kubernetes cluster, services should make use of readiness, liveness and startup probes unless there is a significant reason not to.
129
+
130
+
Probe end points should follow the convention of `healthy` for readiness probes and `healthz` for liveness/startup probes.
131
+
132
+
## Status
133
+
134
+
This standard was first adopted 15 January 2021.
132
135
133
-
Probe end points should follow the convention of `healthy` for readiness probes and `healthz` for liveness probes.
136
+
## Significant changes
137
+
- Entra ID Workload Identities replaces Azure AD Pod Identity for authenticating AKS pods to Azure resources on 7 January 2026
0 commit comments