Skip to content

Optimization Tools and Tweaks

Jump to bottom
Antonios Voulvoulis edited this page Apr 14, 2026 · 2 revisions

Optimization Tools and Tweaks

Optional performance optimizations for NFTBan feed processing and large IP set management.

Table of Contents

Purpose

NFTBan processes threat intelligence feeds and geographic IP lists that can contain hundreds of thousands of entries. Without optimization, these lists are loaded as-is into nftables interval sets, resulting in larger sets, higher memory usage, and slower kernel operations.

This page documents optional tools that reduce list sizes through CIDR aggregation. These tools are not required for normal operation. NFTBan always works without them using safe fallbacks.

CIDR Aggregation

What It Does

CIDR aggregation merges overlapping and adjacent IP entries into compact CIDR blocks.

Before:  192.168.1.0/24, 192.168.1.128/25, 192.168.2.0/24
After:   192.168.1.0/24, 192.168.2.0/24

Before:  10.0.0.0/24, 10.0.1.0/24
After:   10.0.0.0/23

Why It Matters

Metric Without Aggregation With Aggregation
Set size (typical feed) 500K entries 50K-100K entries
nftables memory Higher Lower
Ban operation time (500K set) ~25 seconds ~5-10 seconds
Feed update time Slower Faster

The kernel stores interval sets as red-black trees. Fewer entries = smaller tree = faster lookups and insertions.

Supported Methods

NFTBan automatically selects the best available method. The fallback chain is:

1. aggregate6 (Best)

Full CIDR aggregation for both IPv4 and IPv6. Handles merging, normalization, and overlap absorption.

# IPv4 example
echo -e "10.0.0.0/24\n10.0.1.0/24" | aggregate6 -4
# Output: 10.0.0.0/23

# IPv6 example
echo -e "2001:db8::/48\n2001:db8:1::/48\n2001:db8:2::/48\n2001:db8:3::/48" | aggregate6 -6
# Output: 2001:db8::/46
  • IPv4: Full support
  • IPv6: Full support
  • Binary location: /usr/local/bin/aggregate6

2. netmask (Fallback)

Native system package. Performs basic CIDR normalization for IPv4 only.

netmask -c < ip_list.txt | sort -u > merged.txt
  • IPv4: Supported
  • IPv6: Not supported (IPv6 lists use bash fallback)
  • Package: netmask (Debian, Ubuntu, AlmaLinux via EPEL)

3. Bash Dedup (Minimal)

Built-in fallback. Removes exact duplicates and contained CIDRs. Does not merge adjacent ranges.

  • IPv4: Dedup only (no neighbor merge)
  • IPv6: Dedup only
  • Dependencies: None (built into NFTBan)

Method Comparison

Capability aggregate6 netmask bash
IPv4 merge Yes Partial No
IPv6 merge Yes No No
Adjacent merge (10.0.0.0/24 + 10.0.1.0/24 = /23) Yes No No
Overlap absorption Yes Partial No
Duplicate removal Yes Yes Yes
External dependency Yes (pip3) Yes (package) None

Setup

Check Current Method

nftban status

The output includes the active CIDR method.

Install aggregate6

Run the explicit setup command:

nftban update --setup-cidr-tools

This command:

  1. Detects the Python version and distro
  2. Installs build prerequisites if needed (Python 3.12+ requires gcc and python3-dev/python3-devel)
  3. Installs aggregate6 via pip3
  4. Verifies the tool works with IPv4 and IPv6 test cases
  5. Falls back to netmask or bash if installation fails

The setup never modifies the firewall or interrupts protection.

Manual Install

For environments where the automated setup is not suitable:

# Python 3.12+ (Debian 13, Ubuntu 24.04, Rocky 10)
pip3 install py-radix-sr --break-system-packages
pip3 install aggregate6 --no-deps --break-system-packages

# Python 3.9-3.11 (AlmaLinux 9, RHEL 9)
pip3 install aggregate6

Why two steps on Python 3.12+: The upstream py-radix C extension does not compile against Python 3.12+ headers. py-radix-sr is a maintained fork that fixes this. The --no-deps flag tells pip to use the already-installed fork instead of pulling the broken original.

Prerequisites for Manual Install

Distro Python Packages Needed
Debian 13 3.13 python3-pip python3-dev gcc
Ubuntu 24.04 3.12 python3-pip python3-dev gcc
Rocky 10 3.12 python3-pip python3-devel gcc
AlmaLinux 9 3.9 python3-pip (no build tools needed)

These are not installed automatically by the NFTBan package. They are build-time tools required only for setting up aggregate6.

Verification

After setup, verify the tool works:

# Check which tool is active
nftban status | grep -i cidr

# Test IPv4 aggregation
echo -e "10.0.0.0/24\n10.0.1.0/24" | aggregate6 -4
# Expected: 10.0.0.0/23

# Test IPv6 aggregation
echo -e "2001:db8::/48\n2001:db8:1::/48" | aggregate6 -6
# Expected: 2001:db8::/47

Distro Compatibility

Distro aggregate6 netmask bash Notes
Debian 13 Via pip3 (needs gcc) apt install netmask Built-in Full support
Ubuntu 24.04 Via pip3 (needs gcc) apt install netmask Built-in Full support
Rocky 10 Via pip3 (needs gcc) Not available Built-in No netmask in repos
AlmaLinux 9 Via pip3 (direct) dnf install netmask (EPEL) Built-in Pre-built wheel

Rocky 10 note: netmask is not available in any Rocky 10 repository. aggregate6 is the only CIDR optimization tool on this platform. If aggregate6 is not installed, NFTBan uses bash dedup.

Health Reporting

NFTBan health checks report the active CIDR method.

Severity Levels

Condition Level Exit Code
aggregate6 active OK 0
netmask active (IPv4 fallback) INFO 0
bash fallback active INFO 0
aggregate6 installed but broken WARN 1

Missing CIDR tools are reported as INFO, not ERROR. NFTBan is fully functional without them.

JSON Output

{
  "health": {
    "cidr_aggregation": {
      "tool": "aggregate6",
      "ipv4": true,
      "ipv6": true,
      "status": "protected"
    }
  }
}

When to Enable

Enable CIDR Optimization If

  • Using threat intelligence feeds (nftban feeds enable)
  • Using GeoBan for country-level blocking
  • Managing more than 10,000 banned IPs
  • Running frequent feed updates (hourly or more)
  • Observing slow ban operations on loaded systems

Safe to Skip If

  • Manual bans only (fewer than 1,000 IPs)
  • Testing or development environments
  • No feeds or GeoBan configured
  • System resources are not constrained

Troubleshooting

nftban update --setup-cidr-tools fails

Check prerequisites:

# Debian/Ubuntu
dpkg -l python3-pip python3-dev gcc

# Rocky/Alma
rpm -q python3-pip python3-devel gcc

Install any missing packages, then retry.

aggregate6 installed but not detected

Verify the binary is in PATH:

which aggregate6
# Expected: /usr/local/bin/aggregate6

If installed in a Python user directory, create a symlink:

ln -s ~/.local/bin/aggregate6 /usr/local/bin/aggregate6

py-radix build fails on Python 3.12+

This is expected. The upstream py-radix package has a broken C extension on Python 3.12+.

Fix:

pip3 install py-radix-sr --break-system-packages
pip3 install aggregate6 --no-deps --break-system-packages

Feeds still large after enabling aggregate6

Aggregation reduces overlapping and adjacent entries. If a feed contains dispersed individual IPs with no overlap, the reduction is minimal. This is normal.

Check actual reduction:

# View aggregation log entries
grep "CIDR merge" /var/log/nftban/nftban.log

Example output:

CIDR merge (aggregate6): 500000 -> 48532 (merged 451468 overlaps)

Design Philosophy

NFTBan follows the principle: always work, optimize when available.

  • The firewall never fails because an optimizer is missing
  • Optimization tools are optional enhancements, not requirements
  • The system adapts to the available environment
  • Setup is an explicit admin action, not an implicit side effect of installation

Build tools (gcc, python3-dev) are never pulled automatically by the NFTBan package. They are installed on demand during explicit CIDR tool setup.

References

NFTBan Wiki

Getting Started

Architecture

Modules

Operator Reference

Verification & Trust

Reference

Legal

Clone this wiki locally