CIS Benchmark Level 2 benchmark failure on EKS #540
Description
Package I'm using:
bloodhound/kubelet
What I expected to happen:
After following the instructions in this blog post, I expected to be able to run apiclient report cis -l 2 and pass all the tests.
Configuration
Bottlerocket: v1.40.0 (x86_64)
EKS: v1.31
Here is our bottlerocket configuration
We're using Terraform to template the file, so there are some variables and a base64 encoding function in there
[settings.kubernetes]
api-server = "${api_server}"
cluster-certificate = "${cluster_certificate}"
cluster-name = "${cluster_name}"
node-labels = { pool-name = "${pool_name}" }
cluster-dns-ip = "${dns_cluster_ip}"
[settings.kernel]
lockdown = "integrity"
[settings.kernel.sysctl]
"net.ipv4.conf.all.send_redirects" = "0"
"net.ipv4.conf.default.send_redirects" = "0"
"net.ipv4.conf.all.accept_redirects" = "0"
"net.ipv4.conf.default.accept_redirects" = "0"
"net.ipv6.conf.all.accept_redirects" = "0"
"net.ipv6.conf.default.accept_redirects" = "0"
"net.ipv4.conf.all.secure_redirects" = "0"
"net.ipv4.conf.default.secure_redirects" = "0"
"net.ipv4.conf.all.log_martians" = "1"
"net.ipv4.conf.default.log_martians" = "1"
[settings.kernel.modules.udf]
allowed = false
[settings.kernel.modules.sctp]
allowed = false
[settings.bootstrap-containers.iptables]
source = "public.ecr.aws/bottlerocket/bottlerocket-bootstrap:v0.2.0"
mode = "always"
essential = true
user-data = "${base64encode(<<EOT
#!/usr/bin/env bash
set -xeuo pipefail
dnf install -y iptables-legacy
iptables -F
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP
iptables -I INPUT -p tcp -m tcp --dport 10250 -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A INPUT -s 127.0.0.0/8 -j DROP
iptables -A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p icmp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -m state --state ESTABLISHED -j ACCEPT
ip6tables -F
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP
ip6tables -A INPUT -p tcp --destination-port 10250 -j ACCEPT
ip6tables -A INPUT -i lo -j ACCEPT
ip6tables -A OUTPUT -o lo -j ACCEPT
ip6tables -A INPUT -s ::1 -j DROP
ip6tables -A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
ip6tables -A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT
ip6tables -A OUTPUT -p icmp -m state --state NEW,ESTABLISHED -j ACCEPT
ip6tables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
ip6tables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
ip6tables -A INPUT -p icmp -m state --state ESTABLISHED -j ACCEPT
EOT
)
}"What actually happened:
I encountered one failing test
Test results
$ apiclient report cis -l 2
Benchmark name: CIS Bottlerocket Benchmark
Version: v1.0.0
Reference: https://www.cisecurity.org/benchmark/bottlerocket
Benchmark level: 2
Start time: 2025-06-06T15:11:47.574546770Z
[PASS] 1.1.1.1 Ensure mounting of udf filesystems is disabled (Automatic)
[SKIP] 1.2.1 Ensure software update repositories are configured (Manual)
[PASS] 1.3.1 Ensure dm-verity is configured (Automatic)
[PASS] 1.4.1 Ensure setuid programs do not create core dumps (Automatic)
[PASS] 1.4.2 Ensure address space layout randomization (ASLR) is enabled (Automatic)
[PASS] 1.4.3 Ensure unprivileged eBPF is disabled (Automatic)
[PASS] 1.4.4 Ensure user namespaces are disabled (Automatic)
[PASS] 1.5.1 Ensure SELinux is configured (Automatic)
[PASS] 1.5.2 Ensure Lockdown is configured (Automatic)
[SKIP] 1.6 Ensure updates, patches, and additional security software are installed (Manual)
[PASS] 2.1.1.1 Ensure chrony is configured (Automatic)
[PASS] 3.1.1 Ensure packet redirect sending is disabled (Automatic)
[PASS] 3.2.1 Ensure source routed packets are not accepted (Automatic)
[PASS] 3.2.2 Ensure ICMP redirects are not accepted (Automatic)
[PASS] 3.2.3 Ensure secure ICMP redirects are not accepted (Automatic)
[PASS] 3.2.4 Ensure suspicious packets are logged (Automatic)
[PASS] 3.2.5 Ensure broadcast ICMP requests are ignored (Automatic)
[PASS] 3.2.6 Ensure bogus ICMP responses are ignored (Automatic)
[PASS] 3.2.7 Ensure TCP SYN Cookies is enabled (Automatic)
[PASS] 3.3.1 Ensure SCTP is disabled (Automatic)
[FAIL] 3.4.1.1 Ensure IPv4 default deny firewall policy (Automatic)
[PASS] 3.4.1.2 Ensure IPv4 loopback traffic is configured (Automatic)
[SKIP] 3.4.1.3 Ensure IPv4 outbound and established connections are configured (Manual)
[PASS] 3.4.2.1 Ensure IPv6 default deny firewall policy (Automatic)
[PASS] 3.4.2.2 Ensure IPv6 loopback traffic is configured (Automatic)
[SKIP] 3.4.2.3 Ensure IPv6 outbound and established connections are configured (Manual)
[PASS] 4.1.1.1 Ensure journald is configured to write logs to persistent disk (Automatic)
[PASS] 4.1.2 Ensure permissions on journal files are configured (Automatic)
Passed: 23
Failed: 1
Skipped: 4
Total checks: 28
Compliance check result: FAIL
The IPv4 default deny test failure was unexepcted, since it was applied successfully according to the boot logs. The IPv6 version does pass, which suggested something was overriding the IPv4 rule. This appears to be the result of the kubelet overriding this iptables rule when starting.
Is it possible to pass CIS level 2 with EKS?
How to reproduce the problem:
- Create an EKS cluster running Kubernetes 1.31 and using Bottlerocket 1.40.0.
- Configure it with the additional settings listed in the blog post.
- Connect to the Bottlerocket node and run
apiclient report cis -l 2to see the failure.