Skip to content

Commit 49783fa

Browse files
committed
github-ci: add namespace based IPS tests
Implementing a bridge test with af-packet IPS and a routing test with iptables + nfqueue. Very helpful explanation and guidance on network namespaces can be found here: https://www.redhat.com/en/blog/net-namespaces Sets up 3 network namespaces: 1. client - for running client tools like ping, curl, wget 2. server - for running a server, currently only Caddy 3. dut - for running Suricata, this namespace connects the client and server namespaces Validate IPS operations in 3 ways: 1. check return codes of the client tools 2. check Suricata's IPS stats 3. use tshark to validate expected drops Run Suricata in AF_PACKET IPS mode for both autofp and workers mode. Do the same for NFQUEUE. Tshark's JSON output is used with JQ to validate that pings are dropped. All tests are codecov enabled.
1 parent 2cf9a32 commit 49783fa

6 files changed

Lines changed: 744 additions & 1 deletion

File tree

.github/codecov.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
codecov:
22
require_ci_to_pass: false
33
notify:
4-
after_n_builds: 5
4+
after_n_builds: 6
55

66
coverage:
77
precision: 2

.github/workflows/builds.yml

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1897,6 +1897,132 @@ jobs:
18971897
- run: |
18981898
./qa/unix.sh "suricata-verify/"
18991899
1900+
ubuntu-latest-namespace-ips:
1901+
name: Ubuntu 24.04 (afpacket IPS tests in namespaces)
1902+
runs-on: ubuntu-latest
1903+
needs: [prepare-deps, prepare-cbindgen]
1904+
container:
1905+
image: ubuntu:24.04
1906+
options: --privileged
1907+
steps:
1908+
- name: Cache ~/.cargo
1909+
uses: actions/cache@9255dc7a253b0ccc959486e2bca901246202afeb
1910+
with:
1911+
path: ~/.cargo/registry
1912+
key: cargo-registry
1913+
- name: Determine number of CPUs
1914+
run: echo CPUS=$(nproc --all) >> $GITHUB_ENV
1915+
1916+
- name: Install dependencies
1917+
run: |
1918+
apt update
1919+
apt -y install \
1920+
libpcre2-dev \
1921+
build-essential \
1922+
autoconf \
1923+
automake \
1924+
llvm-19-dev \
1925+
clang-19 \
1926+
git \
1927+
hwloc \
1928+
libhwloc-dev \
1929+
jq \
1930+
inetutils-ping \
1931+
libc++-dev \
1932+
libc++abi-dev \
1933+
libtool \
1934+
libpcap-dev \
1935+
libnet1-dev \
1936+
libyaml-0-2 \
1937+
libyaml-dev \
1938+
libcap-ng-dev \
1939+
libcap-ng0 \
1940+
libmagic-dev \
1941+
libnetfilter-queue-dev \
1942+
libnetfilter-queue1 \
1943+
libnfnetlink-dev \
1944+
libnfnetlink0 \
1945+
libnuma-dev \
1946+
libhiredis-dev \
1947+
libjansson-dev \
1948+
libevent-dev \
1949+
libevent-pthreads-2.1-7 \
1950+
make \
1951+
parallel \
1952+
python3-yaml \
1953+
software-properties-common \
1954+
sudo \
1955+
zlib1g \
1956+
zlib1g-dev \
1957+
exuberant-ctags \
1958+
unzip \
1959+
curl \
1960+
time \
1961+
wget \
1962+
caddy \
1963+
ethtool \
1964+
iproute2 \
1965+
iptables \
1966+
tshark
1967+
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
1968+
- uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
1969+
with:
1970+
name: prep
1971+
path: prep
1972+
# packaged Rust version is too old for coverage, so get from rustup. 1.85.1 matches
1973+
# LLVM 19
1974+
- name: Install Rust
1975+
run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain 1.85.1 -y
1976+
- uses: ./.github/actions/install-cbindgen
1977+
- run: ./autogen.sh
1978+
- run: ./configure --disable-shared --localstatedir=/var --prefix=/usr --sysconfdir=/etc --enable-nfqueue
1979+
env:
1980+
CC: "clang-19"
1981+
CXX: "clang++-19"
1982+
RUSTFLAGS: "-C instrument-coverage"
1983+
CFLAGS: "-fprofile-instr-generate -fcoverage-mapping -O0"
1984+
CXXFLAGS: "-fprofile-instr-generate -fcoverage-mapping -O0"
1985+
- run: make -j ${{ env.CPUS }}
1986+
env:
1987+
CC: "clang-19"
1988+
CXX: "clang++-19"
1989+
RUSTFLAGS: "-C instrument-coverage"
1990+
CFLAGS: "-fprofile-instr-generate -fcoverage-mapping -O0"
1991+
CXXFLAGS: "-fprofile-instr-generate -fcoverage-mapping -O0"
1992+
1993+
- run: |
1994+
./.github/workflows/netns/afp-ips-netns-bridge.sh "2" "workers" ".github/workflows/netns/ips-netns.yaml"
1995+
env:
1996+
LLVM_PROFILE_FILE: "/tmp/afp-ips.profraw"
1997+
- run: llvm-profdata-19 merge -o afp-ips.profdata /tmp/afp-ips.profraw
1998+
1999+
- run: |
2000+
./.github/workflows/netns/afp-ips-netns-bridge.sh "2" "autofp" ".github/workflows/netns/ips-netns.yaml"
2001+
env:
2002+
LLVM_PROFILE_FILE: "/tmp/afp-ips-autofp.profraw"
2003+
- run: llvm-profdata-19 merge -o afp-ips-autofp.profdata /tmp/afp-ips-autofp.profraw
2004+
2005+
- run: |
2006+
./.github/workflows/netns/nfq-ips-netns-route.sh "autofp" ".github/workflows/netns/ips-netns.yaml"
2007+
env:
2008+
LLVM_PROFILE_FILE: "/tmp/nfq-ips.profraw"
2009+
- run: llvm-profdata-19 merge -o nfq-ips.profdata /tmp/nfq-ips.profraw
2010+
2011+
- run: |
2012+
./.github/workflows/netns/nfq-ips-netns-route.sh "workers" ".github/workflows/netns/ips-netns.yaml"
2013+
env:
2014+
LLVM_PROFILE_FILE: "/tmp/nfq-ips-workers.profraw"
2015+
- run: llvm-profdata-19 merge -o nfq-ips-workers.profdata /tmp/nfq-ips-workers.profraw
2016+
2017+
- run: llvm-profdata-19 merge -o combined.profdata afp-ips.profdata nfq-ips.profdata afp-ips-autofp.profdata nfq-ips-workers.profdata
2018+
- run: llvm-cov-19 show ./src/suricata -instr-profile=combined.profdata --show-instantiations --ignore-filename-regex="^/root/.*" > coverage.txt
2019+
- name: Upload coverage to Codecov
2020+
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de
2021+
with:
2022+
fail_ci_if_error: false
2023+
flags: netns
2024+
verbose: true
2025+
19002026
ubuntu-24-04-asan-afpdpdk:
19012027
name: Ubuntu 24.04 (afpacket and dpdk live tests with ASAN)
19022028
runs-on: ubuntu-latest
Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
#!/bin/bash
2+
3+
# Script to test live IPS capabilities for AF_PACKET.
4+
#
5+
# Uses 3 network namespaces:
6+
# - client
7+
# - server
8+
# - dut
9+
#
10+
# Dut is where Suricata will run:
11+
#
12+
# [ client ]$clientif - $dutclientif[ dut ]$dutserverif - $serverif[ server ]
13+
#
14+
# By copying packets between the dut interfaces, Suricata becomes the bridge.
15+
16+
# Call with following arguments:
17+
# 1st: "2" or "3" to indicate the tpacket version.
18+
# 2nd: runmode string (single/autofp/workers)
19+
# 3rd: suricata yaml to use
20+
21+
set -e
22+
set -x
23+
24+
if [ $# -ne "3" ]; then
25+
echo "ERROR call with 3 args: tpacket version (2/3), runmode (single/autofp/workers) and yaml"
26+
exit 1;
27+
fi
28+
29+
TPACKET=$1
30+
RUNMODE=$2
31+
YAML=$3
32+
33+
# dump some info
34+
echo "* printing some diagnostics..."
35+
ip netns list
36+
uname -a
37+
ip r
38+
echo "* printing some diagnostics... done"
39+
40+
clientns=client
41+
serverns=server
42+
dutns=dut
43+
clientip="10.10.10.10/24"
44+
serverip='10.10.10.20/24'
45+
clientif=client
46+
serverif=server
47+
dutclientif=dut_client
48+
dutserverif=dut_server
49+
50+
echo "* removing old namespaces..."
51+
NAMESPACES=$(ip netns list|cut -d' ' -f1)
52+
for NS in $NAMESPACES; do
53+
if [ $NS = $dutns ] || [ $NS = $clientns ] || [ $NS = $serverns ]; then
54+
ip netns delete $NS
55+
fi
56+
done
57+
echo "* removing old namespaces... done"
58+
59+
# remove eve.json from previous run
60+
if [ -f eve.json ]; then
61+
rm eve.json
62+
fi
63+
64+
if [ -e ./rust/target/release/suricatasc ]; then
65+
SURICATASC=./rust/target/release/suricatasc
66+
else
67+
SURICATASC=./rust/target/debug/suricatasc
68+
fi
69+
70+
RES=0
71+
72+
# adding namespaces
73+
echo "* creating namespaces..."
74+
ip netns add $clientns
75+
ip netns add $serverns
76+
ip netns add $dutns
77+
echo "* creating namespaces... done"
78+
79+
#diagnostics output
80+
echo "* list namespaces..."
81+
ip netns list
82+
ip netns exec $clientns ip ad
83+
ip netns exec $serverns ip ad
84+
ip netns exec $dutns ip ad
85+
echo "* list namespaces... done"
86+
87+
# create virtual ethernet link between client-dut and server-dut
88+
# These are not yet mapped to a namespace
89+
echo "* creating virtual ethernet devices..."
90+
ip link add ptp-$clientif type veth peer name ptp-$dutclientif
91+
ip link add ptp-$serverif type veth peer name ptp-$dutserverif
92+
echo "* creating virtual ethernet devices...done"
93+
94+
echo "* list interface in global namespace..."
95+
ip link
96+
echo "* list interface in global namespace... done"
97+
98+
echo "* map virtual ethernet interfaces to their namespaces..."
99+
ip link set ptp-$clientif netns $clientns
100+
ip link set ptp-$serverif netns $serverns
101+
ip link set ptp-$dutclientif netns $dutns
102+
ip link set ptp-$dutserverif netns $dutns
103+
echo "* map virtual ethernet interfaces to their namespaces... done"
104+
105+
echo "* list namespaces and interfaces within them..."
106+
ip netns list
107+
ip netns exec $clientns ip ad
108+
ip netns exec $serverns ip ad
109+
ip netns exec $dutns ip ad
110+
echo "* list namespaces and interfaces within them... done"
111+
112+
# bring up interfaces. Client and server get IP's.
113+
# Disable rx and tx csum offload on all sides.
114+
115+
echo "* setup client interface..."
116+
ip netns exec $clientns ip addr add $clientip dev ptp-$clientif
117+
ip netns exec $clientns ethtool -K ptp-$clientif rx off tx off
118+
ip netns exec $clientns ip link set ptp-$clientif up
119+
echo "* setup client interface... done"
120+
121+
echo "* setup server interface..."
122+
ip netns exec $serverns ip addr add $serverip dev ptp-$serverif
123+
ip netns exec $serverns ethtool -K ptp-$serverif rx off tx off
124+
ip netns exec $serverns ip link set ptp-$serverif up
125+
echo "* setup server interface... done"
126+
127+
echo "* setup dut interfaces..."
128+
ip netns exec $dutns ethtool -K ptp-$dutclientif rx off tx off
129+
ip netns exec $dutns ethtool -K ptp-$dutserverif rx off tx off
130+
ip netns exec $dutns ip link set ptp-$dutclientif up
131+
ip netns exec $dutns ip link set ptp-$dutserverif up
132+
echo "* setup dut interfaces... done"
133+
134+
# set first rule file
135+
cp .github/workflows/netns/drop-icmp.rules suricata.rules
136+
RULES="suricata.rules"
137+
138+
echo "* starting Suricata in the \"dut\" namespace..."
139+
# Start Suricata in the dut namespace, then SIGINT after 240 secords. Will
140+
# close it earlier through the unix socket.
141+
timeout --kill-after=300 --preserve-status 240 \
142+
ip netns exec $dutns \
143+
./src/suricata -c $YAML -l ./ --af-packet -v \
144+
--set default-rule-path=. --runmode=$RUNMODE -S $RULES &
145+
SURIPID=$!
146+
sleep 10
147+
echo "* starting Suricata... done"
148+
149+
echo "* starting tshark on in the server namespace..."
150+
timeout --kill-after=240 --preserve-status 180 \
151+
ip netns exec $serverns \
152+
tshark -i ptp-$serverif -T json > tshark-server.json &
153+
TSHARKSERVERPID=$!
154+
sleep 5
155+
echo "* starting tshark on in the server namespace... done, pid $TSHARKSERVERPID"
156+
157+
echo "* starting Caddy..."
158+
# Start Caddy in the server namespace
159+
timeout --kill-after=240 --preserve-status 120 \
160+
ip netns exec $serverns \
161+
caddy file-server --domain 10.10.10.20 --browse &
162+
CADDYPID=$!
163+
sleep 10
164+
echo "* starting Caddy in the \"server\" namespace... done"
165+
166+
echo "* running curl in the \"client\" namespace..."
167+
ip netns exec $clientns \
168+
curl -O https://10.10.10.20/index.html
169+
echo "* running curl in the \"client\" namespace... done"
170+
171+
echo "* running wget in the \"client\" namespace..."
172+
ip netns exec $clientns \
173+
wget https://10.10.10.20/index.html
174+
echo "* running wget in the \"client\" namespace... done"
175+
176+
ping_ip=$(echo $serverip|cut -f1 -d'/')
177+
echo "* running ping $ping_ip in the \"client\" namespace..."
178+
set +e
179+
ip netns exec $clientns \
180+
ping -c 10 $ping_ip
181+
PINGRES=$?
182+
set -e
183+
echo "* running ping in the \"client\" namespace... done"
184+
185+
# pings should have been dropped, so ping reports error
186+
if [ $PINGRES != 1 ]; then
187+
echo "ERROR ping should have failed"
188+
RES=1
189+
fi
190+
191+
# give stats time to get updated
192+
sleep 10
193+
194+
echo "* shutting down tshark..."
195+
kill -INT $TSHARKSERVERPID
196+
wait $TSHARKSERVERPID
197+
echo "* shutting down tshark... done"
198+
199+
ACCEPTED=$(jq -c 'select(.event_type == "stats")' ./eve.json | tail -n1 | jq '.stats.ips.accepted')
200+
BLOCKED=$(jq -c 'select(.event_type == "stats")' ./eve.json | tail -n1 | jq '.stats.ips.blocked')
201+
KERNEL_PACKETS=$(jq -c 'select(.event_type == "stats")' ./eve.json | tail -n1 | jq '.stats.capture.kernel_packets')
202+
echo "ACCEPTED $ACCEPTED BLOCKED $BLOCKED KERNEL_PACKETS $KERNEL_PACKETS"
203+
204+
if [ $KERNEL_PACKETS -eq 0 ]; then
205+
echo "ERROR no packets captured"
206+
RES=1
207+
fi
208+
if [ $ACCEPTED -eq 0 ]; then
209+
echo "ERROR should have seen non-0 accepted"
210+
RES=1
211+
fi
212+
if [ $BLOCKED -ne 10 ]; then
213+
echo "ERROR should have seen 10 blocked"
214+
RES=1
215+
fi
216+
217+
# validate that we didn't receive pings
218+
SERVER_RECV_PING=$(jq -c '.[]' ./tshark-server.json|jq 'select(._source.layers.icmp."icmp.type"=="8")'|wc -l)
219+
echo "* server pings received check (should be 0): $SERVER_RECV_PING"
220+
if [ $SERVER_RECV_PING -ne 0 ]; then
221+
jq '.[]' ./tshark-server.json | jq 'select(._source.layers.icmp)'
222+
RES=1
223+
fi
224+
echo "* server pings received check... done"
225+
226+
echo "* shutting down..."
227+
kill -INT $CADDYPID
228+
wait $CADDYPID
229+
ip netns exec $dutns \
230+
${SURICATASC} -c "shutdown" /var/run/suricata/suricata-command.socket
231+
wait $SURIPID
232+
echo "* shutting down... done"
233+
234+
echo "* dumping some stats..."
235+
cat ./eve.json | jq -c 'select(.tls)'|tail -n1|jq
236+
cat ./eve.json | jq -c 'select(.stats)|.stats.ips'|tail -n1|jq
237+
cat ./eve.json | jq -c 'select(.stats)|.stats.capture'|tail -n1|jq
238+
cat ./eve.json | jq
239+
echo "* dumping some stats... done"
240+
241+
242+
echo "* done: $RES"
243+
exit $RES
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
drop icmp any any -> any any (itype:8; sid:1;)

0 commit comments

Comments
 (0)