Capture RDMA/InfiniBand traffic with tcpdump, packaged in Docker.
# Clone and install
git clone https://github.com/edmundselliot/rdma-tcpdump.git
cd rdma-tcpdump
sudo ./install.sh
# List available RDMA devices
rdma-tcpdump -D | grep mlx
# Capture traffic on an RDMA device
rdma-tcpdump -ni mlx5_0Installation builds the Docker image before copying the wrapper. The first
build compiles rdma-core, libpcap, and tcpdump, so it may take several minutes;
Docker build progress is printed continuously while it runs. A successful
install leaves rdma-tcpdump ready to run without a first-use build.
BuildKit caches package downloads between builds, fetches independent source
trees in parallel, and compiles only libibverbs plus the mlx5 provider.
All arguments are passed directly to tcpdump:
rdma-tcpdump [tcpdump options]| Flag | Description |
|---|---|
--rebuild |
Force rebuild the Docker image, with full progress output |
--help |
Show help message |
# List all capture devices
rdma-tcpdump -D
# Capture on mlx5_0 with verbose output
rdma-tcpdump -ni mlx5_0 -v
# Capture and write to file
rdma-tcpdump -ni mlx5_0 -w capture.pcap
# Absolute and nested paths work too; the parent directory must exist
rdma-tcpdump -ni mlx5_0 -w /dev/shm/capture.pcap
# Force rebuild (e.g. after updating)
rdma-tcpdump --rebuild -DCapture files named with -w are written directly to the requested host path.
Relative paths are resolved from the directory where rdma-tcpdump is run.
Only the output file's parent directory is mounted into the container. Existing
-C, -G, and -W rotation options continue to create their files in that
host directory.
The image builds libpcap's RDMA sniffer with 16,384 receive entries instead of the upstream default of 128. RDMA packets enter this queue before BPF filtering, so exhausting the default queue can silently lose packets without increasing tcpdump's reported drop count.
The larger queue allocates about 156 MiB of registered receive memory for each RDMA capture.
sudo rm /usr/local/bin/rdma-tcpdump