Skip to content

Commit 623450f

Browse files
Update kubernetes_standards.md (#79)
Co-authored-by: John Watson <john.watson1@defra.gov.uk>
1 parent 1f19a22 commit 623450f

1 file changed

Lines changed: 22 additions & 18 deletions

File tree

docs/standards/kubernetes_standards.md

Lines changed: 22 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,29 @@
44
### Use a managed Kubernetes service
55
Managed Kubernetes services such as Azure Kubernetes Service (AKS) in Azure or Elastic Kubernetes Service (EKS) in AWS are used as opposed to any IaaS Kubernetes implementation.
66

7-
This is because managed Kubernetes services abstract the maintenence and configuration of master nodes to the cloud provider, meaning teams only need to support the worker nodes where services run. Maintaining a full Kubernetes cluster can be very complicated and requires a high level of in depth Kubernetes and networking knowledge which can be a barrier to entry for some teams. Using a managed service significantly reduces this complexity.
7+
This is because managed Kubernetes services abstract the maintenance and configuration of master nodes to the cloud provider, meaning teams only need to support the worker nodes where services run. Maintaining a full Kubernetes cluster can be very complicated and requires a high level of in depth Kubernetes and networking knowledge which can be a barrier to entry for some teams. Using a managed service significantly reduces this complexity.
88

99
### Use Helm for packaging deployments
1010
[Helm](https://helm.sh/) is a tool to bundle individual Kubernetes configuration files into single deployable packages. This significantly reduces the complexity of configuring a cluster and deploying applications to it.
1111

12-
Helm 3 must be used and teams should never use Helm 2 due to security risk introduced by running Tiller in a cluster.
12+
Helm >=3 must be used and teams should never use Helm 2 due to security risk introduced by running Tiller in a cluster.
1313

14-
Teams are also encouraged to use a Helm Library chart to reduce duplication of Helm charts across multiple services. Such as [this one](https://github.com/DEFRA/ffc-helm-library) used by the Future Farming and Countryside Programme (FFC).
14+
Teams are also encouraged to use a Helm Library chart to reduce duplication of Helm charts across multiple services. Such as [this one](https://github.com/DEFRA/ffc-helm-library) used by the Farming and Countryside Programme (FCP).
1515

1616
### Use ConfigMaps for configuration
1717
Configuration for an application running in a pod should be passed to the pod via a `ConfigMap` Kubernetes resource.
1818
The `ConfigMap` is more flexible than just using environment variables alone, and as well as supporting file based values, allows decoupling of pod definitions from configuration definitions.
1919

20-
Environment specific values should be overriden during the Helm deployment.
20+
Environment specific values should be overridden during the Helm deployment.
2121

2222
Sensitive values should never be passed to a `ConfigMap`.
2323

2424
### Secrets
25-
Where possible, secrets should not be stored within an application or Kubernetes pod. Instead, when communicating with supported cloud infrastructure, clusters should use [AAD Pod Identity](https://github.com/Azure/aad-pod-identity) in Azure or [IAM role for Service Accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) in AWS.
25+
Where possible, secrets should not be stored within an application or Kubernetes pod. Instead, when communicating with supported cloud infrastructure, clusters should use [Entra ID Workload Identities](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet) in Azure or [IAM role for Service Accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) in AWS.
2626

2727
When secrets in a pod are unavoidable, for example when a third party API key is needed, secrets should be injected into pods during deployment.
2828

29-
**Note** the precise mechanism for managing this is still being reviewed with in a collaboration between Cloud Technology Working Group (CTWG), Future Farming and Countryside Programme (FFC) and Europe and Trade Delivery Portfolio (EuTDP).
30-
31-
This document will be updated when an agreement is reached.
32-
33-
The Kubernetes `Secrets` resource type must not be used as data is only Base64 encrypted.
29+
> Note: the Kubernetes `Secrets` resource type stores data as Base64 encoded as opposed to encryption. Ensure appropriate access controls are in place for reading cluster secrets.
3430
3531
### Labels
3632
Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system.
@@ -42,7 +38,7 @@ In order to take full advantage of using labels, they should be applied on every
4238
#### Required labels
4339
Each Helm chart templated resource should have the below labels. Example placeholders are provided for values.
4440

45-
```
41+
```yaml
4642
metadata:
4743
labels:
4844
app: {{ quote .Values.namespace }}
@@ -60,7 +56,8 @@ metadata:
6056

6157
### Selectors
6258
Services selectors should be matched by app and name. Selectors should be consistent otherwise updates to Helm charts will be rejected.
63-
```
59+
60+
```yaml
6461
selector:
6562
app: {{ quote .Values.name }}
6663
app.kubernetes.io/name: {{ quote .Values.name }}
@@ -83,7 +80,7 @@ Clusters will limit available resources within a namespace using a `resourceQuot
8380

8481
An example `ResourceQuota` definition is included below
8582

86-
```
83+
```yaml
8784
apiVersion: v1
8885
kind: ResourceQuota
8986
metadata:
@@ -99,9 +96,9 @@ spec:
9996
### Pod priority
10097
Kubernetes Pods can have priority levels. Priority indicates the importance of a pod relative to other pods. If a pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority pods to make scheduling of the pending pod possible.
10198

102-
In the event of over utilisation of a cluster, Kubernetes will start to kill lower priorty pods first to maintain stability.
99+
In the event of over utilisation of a cluster, Kubernetes will start to kill lower priority pods first to maintain stability.
103100

104-
Clusters should include pod priority classes that teams can consume based on their service needs. The below gives examples of the Pod Priorty classes available in FFC clusters.
101+
Clusters should include pod priority classes that teams can consume based on their service needs. The below gives examples of the Pod Priority classes available in FFC clusters.
105102

106103
#### High (1000)
107104
Reserved primarily for customer facing or critical workload pods.
@@ -114,7 +111,7 @@ For pods where downtime is more tolerable.
114111

115112
Below is a full example of a `priorityClass` resource definition for reference.
116113

117-
```
114+
```yaml
118115
apiVersion: scheduling.k8s.io/v1
119116
kind: PriorityClass
120117
metadata:
@@ -128,6 +125,13 @@ description: "This priority class should be used for most services"
128125
In the event a cluster has to make a choice between killing one of two services sharing the same priority level, the resource profile configuration will influence which is killed.
129126

130127
### Probes
131-
To increase the stability and predicatability of a Kubernetes cluster, services should make use of both readiness and liveness probes unless there is a significant reason not to.
128+
To increase the stability and predictability of a Kubernetes cluster, services should make use of readiness, liveness and startup probes unless there is a significant reason not to.
129+
130+
Probe end points should follow the convention of `healthy` for readiness probes and `healthz` for liveness/startup probes.
131+
132+
## Status
133+
134+
This standard was first adopted 15 January 2021.
132135

133-
Probe end points should follow the convention of `healthy` for readiness probes and `healthz` for liveness probes.
136+
## Significant changes
137+
- Entra ID Workload Identities replaces Azure AD Pod Identity for authenticating AKS pods to Azure resources on 7 January 2026

0 commit comments

Comments
 (0)