feat: v1.95 — smoke/selftest split + G20 CI gate#453
Closed
Conversation
Extends v1.94 smoke framework with: - prereqs.go: Reusable prerequisite evaluators (binary, file, systemd, daemon_running, module_enabled, http_endpoint, config_key) - assertions.go: Contract-safe assertion helpers (JSONValid, JSONPathExists, OutputContains, MetricFamiliesPresent, NoFatalPatterns) - tests_modules.go: Module-gated smoke tests for ddos, portscan, botguard, loginmon — SKIP when module disabled (not FAIL) - Extended SmokeTest struct: Module, Assertions, DeepOnly, CIEnabled fields - CLI: --module=MODULE and --deep flags - RunOptions replaces simple group string Module tests do NOT claim enforcement — only that commands run without fatal runtime errors when module is enabled. Truth comes from validator. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split the two different testing concepts into distinct commands: nftban smoke = registry-driven, non-destructive, CI-safe - Go-based (internal/smoke/) - PASS/FAIL/SKIP semantics - --json, --group, --module, --deep flags - Safe for routine checks and fleet monitoring nftban selftest = extended system validation with controlled state changes - Shell-based (tests/selftest.sh) - Ban/unban lifecycle, whitelist mutation, port lifecycle - Intended for deep verification and troubleshooting - May temporarily modify firewall state Changes: - cmd_smoke.sh: rewritten to pure Go pass-through (898→70 LOC) - cmd_selftest.sh: NEW — old smoke shell suite reclassified - smoke_test.sh → selftest.sh (renamed) - test_module_smoke.sh → test_module_selftest.sh (renamed) - commands.registry.yml: smoke updated, selftest added - CONTRIBUTING.md: testing docs updated - nftban dispatcher: selftest command registered - wiki FHS page: test dir description updated - scripts/test_installation.sh: reference updated Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
V1 (HIGH): Old smoke subcommands (run, lifecycle, verify, etc.) now print migration message directing to `nftban selftest`. Prevents silent breakage for operators upgrading from v1.94. V3 (MEDIUM): G20 Smoke Gate wired into ci-smoke.yml. Runs nftban-core smoke --json, validates JSON, fails on FAIL count > 0. SKIPs (expected in CI — no daemon/systemd) do not cause failure. Ref: V195_SMOKE_SELFTEST_AUDIT.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
V4 (P0): ci-architecture.yml:360 referenced deleted test_module_smoke.sh → updated to test_module_selftest.sh V5 (P0): cmd_selftest.sh:114 referenced deleted smoke_test.sh → updated to selftest.sh Also: docs/BUILD_STATUS.md G8-4 reference updated. Ref: V195_SMOKE_SELFTEST_AUDIT.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
T2 (health --json): requires nftban-validate binary. Without it, health falls back to shell diagnostics with undocumented exit codes (4, 7 observed in CI). This is a genuine prerequisite — health delegates to the Go validator for its truth contract. T3 (status): requires nftband.service running. Status queries the live system — without daemon, exit codes are environment-dependent. In CI (no validator binary, no daemon): T2 and T3 SKIP. On live hosts (validator present, daemon running): strict [0,1,2]. No exit code widening. No fake prerequisites. Real dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
G15 registry parity gate requires all registry commands in completion. selftest was added to registry but missing from bash-completion. Also updated smoke completions from old subcommands to new flags (--json, --group, --module, --deep). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
D1 (nftband service): prereq was Type:"binary" Name:"systemctl" which passes in CI even though daemon isn't running. Changed to PrereqDaemonRunning — correctly SKIPs when daemon absent. M1 (/metrics reachable): prereq was Type:"daemon" (unrecognized by prereqs.go, fell through to default:true). Changed to PrereqDaemonRunning — correctly SKIPs when daemon absent. These are real prerequisites: /metrics endpoint requires daemon, and D1 tests daemon state which requires daemon to exist. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return err == nil | ||
|
|
||
| case PrereqDaemonRunning: | ||
| out, err := exec.Command("systemctl", "is-active", p.Name).Output() |
| if err != nil { | ||
| return false | ||
| } | ||
| resp.Body.Close() |
Owner
Author
|
Final recreate — alerts dismissed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v1.95 release. All audit violations fixed. Lab tested.
🤖 Generated with Claude Code