Runtime Configuration Auditing and Drift Detection

[JamesEjembi]

Problem Statement

Production configuration drifts from expected values during incident response or manual fixes, creating silent outages that are hard to diagnose. Implement runtime config auditing that snapshots current config at regular intervals, compares to a committed baseline, and alerts on drift.

Technical Bounds

• Snapshot interval: 5 minutes for all services.
• Baseline: committed config files in repository (config/*.baseline.json).
• Drift types: value change, added key, removed key, type change.
• Alert severity: critical for security-related keys (TLS, auth), warning for performance keys.
• Alert channel: PagerDuty for critical, Slack for warning.
• Dashboard: Grafana panel showing drift history with per-key timeline.

Steps

1. Implement config snapshot collector: GET /config/snapshot on each service, returns full config.
2. Build diff comparator: compare snapshot against baseline, categorize drifts by type and severity.
3. Store drift events in PostgreSQL config_drift_events table.
4. Implement alert routing: PagerDuty API for critical drifts, Slack webhook for warnings.
5. Build Grafana dashboard: per-service drift count over time, latest drifts table.
6. Add auto-remediation: for known safe drifts (auto-scaled values), update baseline automatically with annotation.

[darcszn]

Hi, I'd like to take this on.
I've looked into the issue and have a clear idea of what's causing it.
I'll dig in, write a fix, and make sure it's covered with tests if applicable.
Can I be assigned?

[grantfox-oss]

🦊 GrantFox — @darcszn has been assigned to this issue as part of the Third Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #200)
2. Your PR will be reviewed by the VeriNode-Labs maintainers

Good luck! Track your progress on GrantFox.

[grantfox-oss]

🎉 This issue has been marked as completed on GrantFox as part of the Third Campaign campaign!

@darcszn's PR #240 was approved and merged by @JamesEjembi.

🏆 @darcszn: You earned 40 FoxPoints for this contribution! Your current tier: Builder (1,585 total points). Track your full progress on GrantFox.

👏 Great work, @darcszn! Keep contributing to VeriNode-Labs.