Skip to content

Commit 302bf3b

Browse files
authored
feat: The traffic in transit between the audit webhook service in the cluster and the components that send audit events to it is encrypted using HTTPS in the kube-audit sub-module. The module will run a script to execute steps mentioned [here](https://cloud.ibm.com/docs/openshift?topic=openshift-health-audit#secure-setup) to generate a private key and approve the certificate using kubectl. To opt out of this and use HTTP instead (not recommended) you can set enable_https_traffic to false.<br>- NOTE: This will enabled by default in the Standard - Integrated setup with configurable services DA variation. Users upgrading to a previous version of the DA will see an update in place which will re-create the kube secret and restart the kube-audit pods to apply the HTTPS configuration (#914)
1 parent 0eae26e commit 302bf3b

File tree

12 files changed

+178
-6
lines changed

12 files changed

+178
-6
lines changed

examples/advanced/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,5 +290,5 @@ module "logs_agents" {
290290
value = module.ocp_base.cluster_id
291291
}]
292292
# example of how to add only kube-audit log source path
293-
logs_agent_selected_log_source_paths = ["/var/log/audit/*.log"]
293+
logs_agent_system_logs = ["/var/log/audit/*.log"]
294294
}

ibm_catalog.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -937,6 +937,10 @@
937937
"key": "enable_kube_audit",
938938
"hidden": true
939939
},
940+
{
941+
"key": "enable_kube_audit_https_traffic",
942+
"hidden": true
943+
},
940944
{
941945
"key": "audit_deployment_name",
942946
"hidden": true

modules/kube-audit/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ No modules.
7070
| Name | Type |
7171
|------|------|
7272
| [helm_release.kube_audit](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
73+
| [null_resource.enable_https_traffic](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
7374
| [null_resource.set_audit_log_policy](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
7475
| [null_resource.set_audit_webhook](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
7576
| [terraform_data.install_required_binaries](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
@@ -89,6 +90,7 @@ No modules.
8990
| <a name="input_cluster_config_endpoint_type"></a> [cluster\_config\_endpoint\_type](#input\_cluster\_config\_endpoint\_type) | Specify which type of endpoint to use for for cluster config access: 'default', 'private', 'vpe', 'link'. 'default' value will use the default endpoint of the cluster. | `string` | `"default"` | no |
9091
| <a name="input_cluster_id"></a> [cluster\_id](#input\_cluster\_id) | The ID of the cluster to deploy the log collection service in. | `string` | n/a | yes |
9192
| <a name="input_cluster_resource_group_id"></a> [cluster\_resource\_group\_id](#input\_cluster\_resource\_group\_id) | The resource group ID of the cluster. | `string` | n/a | yes |
93+
| <a name="input_enable_https_traffic"></a> [enable\_https\_traffic](#input\_enable\_https\_traffic) | When set to true, the traffic in transit between the audit webhook service in the cluster and the components that send audit events to it is encrypted using HTTPS. This automates the steps mentioned [here](https://cloud.ibm.com/docs/openshift?topic=openshift-health-audit#secure-setup). Certificate rotation still requires manual intervention to replace the secret and restart the deployment. | `bool` | `true` | no |
9294
| <a name="input_ibmcloud_api_key"></a> [ibmcloud\_api\_key](#input\_ibmcloud\_api\_key) | The IBM Cloud api key to generate an IAM token. | `string` | n/a | yes |
9395
| <a name="input_install_required_binaries"></a> [install\_required\_binaries](#input\_install\_required\_binaries) | When set to true, a script will run to check if `kubectl` and `jq` exist on the runtime and if not attempt to download them from the public internet and install them to /tmp. Set to false to skip running this script. | `bool` | `true` | no |
9496
| <a name="input_region"></a> [region](#input\_region) | The IBM Cloud region where the cluster is provisioned. | `string` | n/a | yes |

modules/kube-audit/helm-charts/kube-audit/templates/deployment.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ spec:
2121
imagePullPolicy: Always
2222
ports:
2323
- containerPort: 3000
24+
- containerPort: 4443
2425
securityContext:
2526
allowPrivilegeEscalation: false
2627
runAsNonRoot: true
@@ -29,3 +30,12 @@ spec:
2930
- ALL
3031
seccompProfile:
3132
type: RuntimeDefault
33+
volumeMounts:
34+
- mountPath: /certificates
35+
name: certificates
36+
readOnly: true
37+
volumes:
38+
- name: certificates
39+
secret:
40+
secretName: "{{ .Values.metadata.name }}-secret"
41+
optional: true

modules/kube-audit/helm-charts/kube-audit/templates/network-policy.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ spec:
1313
- ports:
1414
- protocol: TCP
1515
port: 3000
16+
- protocol: TCP
17+
port: 4443
1618
from:
1719
- namespaceSelector:
1820
matchLabels:

modules/kube-audit/helm-charts/kube-audit/templates/service.yaml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,12 @@ spec:
99
selector:
1010
app: "{{ .Values.metadata.name }}"
1111
ports:
12-
- protocol: TCP
12+
- name: http
13+
protocol: TCP
1314
port: 80
1415
targetPort: 3000
16+
- name: https
17+
protocol: TCP
18+
port: 443
19+
targetPort: 4443
1520
type: ClusterIP

modules/kube-audit/main.tf

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ resource "terraform_data" "install_required_binaries" {
1010
audit_namespace = var.audit_namespace
1111
audit_webhook_listener_image = var.audit_webhook_listener_image
1212
audit_webhook_listener_image_tag_digest = var.audit_webhook_listener_image_tag_digest
13+
enable_https_traffic = var.enable_https_traffic
1314
}
1415
provisioner "local-exec" {
1516
command = "${path.module}/scripts/install-binaries.sh ${local.binaries_path}"
@@ -100,14 +101,30 @@ resource "helm_release" "kube_audit" {
100101
}
101102
}
102103

104+
resource "null_resource" "enable_https_traffic" {
105+
depends_on = [terraform_data.install_required_binaries, helm_release.kube_audit]
106+
count = var.enable_https_traffic ? 1 : 0
107+
triggers = {
108+
enable_https_traffic = var.enable_https_traffic
109+
}
110+
provisioner "local-exec" {
111+
command = "${path.module}/scripts/https_audit.sh ${var.audit_namespace} ${var.audit_deployment_name} ${var.audit_deployment_name}-secret"
112+
interpreter = ["/bin/bash", "-c"]
113+
environment = {
114+
KUBECONFIG = data.ibm_container_cluster_config.cluster_config.config_file_path
115+
}
116+
}
117+
}
118+
119+
103120
# wait for the kube-audit resources.
104121
resource "time_sleep" "wait_for_kube_audit" {
105-
depends_on = [helm_release.kube_audit]
122+
depends_on = [null_resource.enable_https_traffic]
106123
create_duration = "60s"
107124
}
108125

109126
locals {
110-
audit_server = "https://127.0.0.1:2040/api/v1/namespaces/${var.audit_namespace}/services/${var.audit_deployment_name}-service/proxy/post"
127+
audit_server = var.enable_https_traffic ? "https://127.0.0.1:2040/api/v1/namespaces/${var.audit_namespace}/services/https:${var.audit_deployment_name}-service:https/proxy/post" : "https://127.0.0.1:2040/api/v1/namespaces/${var.audit_namespace}/services/http:${var.audit_deployment_name}-service:http/proxy/post"
111128
}
112129

113130
# see [issue](https://github.com/IBM-Cloud/terraform-provider-ibm/issues/6107)
@@ -118,7 +135,8 @@ locals {
118135
resource "null_resource" "set_audit_webhook" {
119136
depends_on = [terraform_data.install_required_binaries, time_sleep.wait_for_kube_audit]
120137
triggers = {
121-
audit_log_policy = var.audit_log_policy
138+
audit_log_policy = var.audit_log_policy
139+
enable_https_traffic = var.enable_https_traffic
122140
}
123141
provisioner "local-exec" {
124142
command = "${path.module}/scripts/set_webhook.sh ${var.region} ${var.use_private_endpoint} ${var.cluster_config_endpoint_type} ${var.cluster_id} ${var.cluster_resource_group_id} ${var.audit_log_policy != "default" ? "verbose" : "default"} ${local.binaries_path}"
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
NAMESPACE=$1
5+
DEPLOYMENT=$2
6+
SERVICE="${DEPLOYMENT}-service"
7+
CSR_NAME="${SERVICE}.${NAMESPACE}"
8+
SECRET_NAME=${3:-"audit-webhook"}
9+
# The binaries downloaded by the install-binaries script are located in the /tmp directory.
10+
export PATH=$PATH:${4:-"/tmp"}
11+
12+
function check_kubectl_cli() {
13+
if ! command -v kubectl &>/dev/null; then
14+
echo "Error: kubectl is not installed. Exiting."
15+
exit 1
16+
fi
17+
}
18+
19+
check_kubectl_cli
20+
21+
echo "Waiting for Service ClusterIP..."
22+
cluster_ip=$(
23+
kubectl wait \
24+
--for jsonpath='{.spec.clusterIP}' \
25+
--namespace "${NAMESPACE}" \
26+
--output jsonpath='{.spec.clusterIP}' \
27+
--timeout 5m \
28+
svc/"${SERVICE}"
29+
)
30+
31+
echo "Cluster IP detected: ${cluster_ip}"
32+
33+
function sign_certificate() {
34+
echo "Generating private key..."
35+
SERVER_KEY="$(openssl genrsa 4096)"
36+
37+
echo "Generating CSR..."
38+
SERVER_CSR="$(
39+
openssl req -new \
40+
-key <(printf '%s\n' "${SERVER_KEY}") \
41+
-config <(
42+
cat <<EOF
43+
[ req ]
44+
default_bits = 2048
45+
prompt = no
46+
default_md = sha256
47+
req_extensions = req_ext
48+
distinguished_name = dn
49+
50+
[ dn ]
51+
CN = system:node:${SERVICE}.${NAMESPACE}.svc
52+
O = system:nodes
53+
54+
[ req_ext ]
55+
subjectAltName = @alt_names
56+
57+
[ alt_names ]
58+
DNS.1 = ${SERVICE}.${NAMESPACE}.svc.cluster.local
59+
DNS.2 = ${SERVICE}.${NAMESPACE}.svc
60+
IP.1 = ${cluster_ip}
61+
EOF
62+
)
63+
)"
64+
65+
echo "Submitting Kubernetes CSR..."
66+
kubectl apply -f - <<EOF
67+
apiVersion: certificates.k8s.io/v1
68+
kind: CertificateSigningRequest
69+
metadata:
70+
name: ${CSR_NAME}
71+
spec:
72+
request: $(printf '%s' "${SERVER_CSR}" | base64 | tr -d '\n')
73+
signerName: kubernetes.io/kubelet-serving
74+
usages:
75+
- digital signature
76+
- key encipherment
77+
- server auth
78+
EOF
79+
80+
echo "Approving CSR..."
81+
kubectl certificate approve "${CSR_NAME}"
82+
83+
echo "Waiting for signed certificate..."
84+
kubectl wait \
85+
--for jsonpath='{.status.certificate}' \
86+
--timeout 5m \
87+
csr/"${CSR_NAME}"
88+
89+
SERVER_CERT="$(
90+
kubectl get csr/"${CSR_NAME}" \
91+
-o jsonpath='{.status.certificate}' | base64 --decode
92+
)"
93+
}
94+
95+
sign_certificate
96+
97+
echo "Creating or replacing TLS secret..."
98+
kubectl delete secret "${SECRET_NAME}" \
99+
--namespace "${NAMESPACE}" \
100+
--ignore-not-found
101+
102+
kubectl create secret tls "${SECRET_NAME}" \
103+
--namespace "${NAMESPACE}" \
104+
--cert <(printf '%s\n' "${SERVER_CERT}") \
105+
--key <(printf '%s\n' "${SERVER_KEY}")
106+
107+
echo "Restarting deployment..."
108+
kubectl rollout restart \
109+
--namespace "${NAMESPACE}" \
110+
deploy/"${DEPLOYMENT}"
111+
112+
echo "Waiting for rollout to complete..."
113+
kubectl rollout status \
114+
--timeout 1m \
115+
--namespace "${NAMESPACE}" \
116+
deploy/"${DEPLOYMENT}"

modules/kube-audit/variables.tf

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,3 +109,10 @@ variable "install_required_binaries" {
109109
description = "When set to true, a script will run to check if `kubectl` and `jq` exist on the runtime and if not attempt to download them from the public internet and install them to /tmp. Set to false to skip running this script."
110110
nullable = false
111111
}
112+
113+
variable "enable_https_traffic" {
114+
type = bool
115+
default = true
116+
description = "When set to true, the traffic in transit between the audit webhook service in the cluster and the components that send audit events to it is encrypted using HTTPS. This automates the steps mentioned [here](https://cloud.ibm.com/docs/openshift?topic=openshift-health-audit#secure-setup). Certificate rotation still requires manual intervention to replace the secret and restart the deployment."
117+
nullable = false
118+
}

solutions/fully-configurable/main.tf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -310,4 +310,5 @@ module "kube_audit" {
310310
audit_deployment_name = var.audit_deployment_name
311311
audit_webhook_listener_image = var.audit_webhook_listener_image
312312
audit_webhook_listener_image_tag_digest = var.audit_webhook_listener_image_tag_digest
313+
enable_https_traffic = var.enable_kube_audit_https_traffic
313314
}

0 commit comments

Comments
 (0)