Large Set Management

Large Set Management

Added in: v1.32.0 Scope: Adaptive counting, cached metrics, global nft operation lock

---

Overview

When nftables sets grow beyond 50,000 entries (common with threat intelligence feeds and geographic blocking), the cost of kernel operations increases significantly. NFTBan v1.32.0 introduces a smart counting architecture that separates enforcement (exact, immediate, kernel-backed) from observability (cached, adaptive, daemon-backed).

Core principle: Packets need exact kernel state. Humans do not.

---

The Problem (Pre-v1.32.0)

Every 60 seconds, the metrics exporter called nft list set to count elements. On a 500K-entry interval set, this operation:

• Takes 15-25 seconds
• Consumes 100% of one CPU core
• Runs concurrently with feed sync writes (no serialization)
• Results in system freeze and SSH connection drops

Additionally, 8 separate nft list set calls ran per collection cycle with no locking, while feed sync was writing to the same set.

---

Scale Levels

NFTBan classifies each set independently based on element count:

Level	Set Size	Exporter Interval	Reconciliation	Display
NORMAL	0 - 9,999	60s	every 1h	exact
LARGE	10,000 - 49,999	60s	every 2h	exact
VERY_LARGE	50,000 - 99,999	120s	every 4h	exact
HUGE	100,000 - 249,999	300s	every 6h	rounded (~NNK)
EXTREME	250,000 - 499,999	600s	every 6h	rounded (~NNNK)
CRITICAL_SCALE	500,000+	900s	every 6h + after sync	rounded (~NNNK+)

The global scale is the maximum across all sets, used for system-wide timer decisions.

---

Architecture

Enforcement Path (unchanged):
  nftban ban → daemon → nft add element → kernel
  Exact. Immediate. No rounding. No delays.

Monitoring Path (new):
  daemon maintains in-memory atomic counters
    → updated on every add/delete/flush/replace
    → debounced write to /run/nftban/set_counts.json (every 10s)

  Exporters/status/UI read cache file
    → O(1) file read, no kernel call
    → Adaptive refresh interval based on scale level

  Reconciliation (rare):
    → on startup, after feed sync, manual verify
    → single set at a time, with exclusive lock
    → background timer: max every 6h for HUGE+ sets

---

Cache File

Path: /run/nftban/set_counts.json

{
  "timestamp": "2026-03-21T16:30:00Z",
  "daemon_pid": 392731,
  "scale_mode": "CRITICAL_SCALE",
  "sets": {
    "blacklist_ipv4": {
      "count": 437446,
      "scale": "CRITICAL_SCALE",
      "display": "~437K",
      "last_reconciled": "2026-03-21T10:09:00Z",
      "trend": "+2341 in last 1h"
    },
    "whitelist_ipv4": {
      "count": 9,
      "scale": "NORMAL",
      "display": "9",
      "last_reconciled": "2026-03-21T10:09:00Z",
      "trend": "stable"
    }
  }
}

Written by daemon (atomic rename, max once per 10 seconds). Read by all consumers (exporters, CLI, UI).

---

Global nft Operation Lock

Path: /run/nftban/nft_operations.lock Mechanism: flock(2) — same syscall used by both shell and Go

Operation	Lock type	Timeout
nft add/delete element	Exclusive	30s
nft flush/replace set	Exclusive	30s
Reconciliation (nft list set)	Exclusive	30s
Rare direct kernel reads	Shared	5s
Exporter cache reads	None needed	-

Exporters that read from the cache file do not need any lock.

---

CLI Flags

nftban stats               # Default: rounded display on HUGE+ sets
nftban stats --exact       # Exact daemon counter (O(1), always safe)
nftban stats --verify      # Kernel reconciliation (warns about cost on HUGE+)
nftban verify              # Full verification of all sets against kernel

---

Display Examples

NORMAL:     23,412 banned
LARGE:      47,891 banned
HUGE:       ~104K banned [HUGE]
EXTREME:    ~312K banned [EXTREME]
CRITICAL:   ~437K banned [CRITICAL_SCALE]

Machine-readable output (JSON, Prometheus, Zabbix) always returns exact daemon counter values, never rounded.

---

Prometheus Metrics

nftban_set_elements{family="ipv4",set="blacklist"} 437446
nftban_set_scale{family="ipv4",set="blacklist"} 5
nftban_set_last_reconciled_seconds{family="ipv4",set="blacklist"} 3600
nftban_set_counter_source{family="ipv4",set="blacklist"} 1

Counter source: 0 = kernel, 1 = daemon counter, 2 = cache file.

---

Reconciliation

The daemon verifies its counters against kernel state at specific triggers:

Trigger	Frequency
Daemon startup	Once (full reconciliation)
After feed sync	After each sync completes
After geoban sync	After each sync completes
Background timer	Every 1-6h (scale-dependent)
Manual nftban verify	On demand
Crash recovery	Once on restart

If reconciliation finds a delta > 1% of set size, a WARNING is logged (possible external modification).

---

Configuration

No configuration required. Scale levels and adaptive timers are automatic based on set size. The system transitions seamlessly between modes as sets grow or shrink.

---

Troubleshooting

Cache file not updating

• Check daemon is running: systemctl status nftband
• Check file age: stat /run/nftban/set_counts.json
• Force refresh: nftban verify

Exporter shows stale data

• Check nftban_set_last_reconciled_seconds metric
• If daemon PID in cache doesn't match running daemon, restart daemon
• If cache age exceeds 2x expected interval, investigate daemon health

Scale banner showing on status

• This is normal for sets > 100K entries
• Use nftban stats --exact for exact daemon counter
• Use nftban stats --verify for kernel-verified count (expensive on large sets)

---

See Also

• Optimization Tools and Tweaks — CIDR aggregation and feed optimization
• Performance Benchmarks — IPC latency and scalability
• Timer Schedule — Systemd timer architecture
• Metrics Architecture — Metrics pipeline and exporters