c-06, c-07, c-08

[alix-graylog]

Summary

Makes the chart highly available by default (soft pod anti-affinity on all three tiers, PodDisruptionBudgets enabled, MongoDB switched to a production topology) and fixes two root-password bugs that made the admin credentials unusable after the first release.

Details

HA defaults (C-08)

• Soft podAntiAffinity (preferredDuringSchedulingIgnoredDuringExecution, topologyKey: kubernetes.io/hostname) on Graylog, Data Node and MongoDB. Soft rather than hard so single-node clusters still schedule. Setting graylog.affinity / datanode.affinity replaces the default.
• The MongoDB term selects app: <crName>-svc, the only label the MongoDB Community operator applies to its pods. Selecting on the chart's own labels matches zero pods and silently disables spreading.
• Both PodDisruptionBudgets default to enabled: true. Added the missing datanode.enabled guard to the Data Node PDB.

MongoDB topology (C-07)

• Default is now replicas: 3, arbiters: 0. PSA (2 data + 1 arbiter) stalls w:majority writes when a data member is down; it remains available as a documented cost-optimized opt-in.
• Fixed an inverted comparison in NOTES.txt (gt 1 $mongoReplicaCount) that was true only at replicas: 0, so the topology description never rendered -Secondary. The same argument-order bug in the odd-member warning is fixed too.

Root password (new)

• NOTES.txt printed a freshly generated password on every upgrade while secrets.yaml kept the stored hash, so from revision 2 onward the chart displayed a password that had never worked.
• graylog.config.rootPassword was ignored on upgrade — the restore overwrote the SHA unconditionally, leaving no working way to change the root password on an existing release.
• Both fixed with a graylog.storedRootPasswordSha helper: it returns the stored hash, or "" when a password is explicitly configured or lookup finds nothing. secrets.yaml uses it as the single restore source and NOTES.txt consults it before displaying anything. Corrected the ADDITIONAL NOTES text that claimed a generated password "will NOT persist across upgrades" — untrue since the hash is restored.

External secret guard (C-06)

• Template-time fail when global.existingSecretName is combined with mongodb.communityResource.enabled=true. That pairing skips backup-secret creation while the MongoDB CR still references it, so the operator hangs forever with no clear error.
• Gated the MongoDB connectivity test hook on chart-managed MongoDB, and added examples/values-existing-secret-external-mongodb.yaml.

Docs

• New README sections: "MongoDB Topology", "High Availability Defaults", and "Data Node Replicas and Data Redundancy" — the last covering elasticsearch_replicas and index-set replicas, since with 0-replica index sets a single Data Node loss is permanent data loss rather than an outage.

Linked issues

This fixes #116 (C-06) and partially addresses #14 (C-08 anti-affinity and PDB defaults).
#126 C-07 and the root-password bugs have no issue open yet.

PR Checklist

Please check the items that apply to your change.

• Tests added/updated
• Documentation updated
• This PR includes a new feature
• This PR includes a bugfix
• This PR includes a refactor

Testing Checklist

Static Validation

• Linter check passes: helm lint ./charts/graylog
• Helm renders local template sucessfully: helm template graylog ./charts/graylog --validate

Installation

• Fresh installation completes successfully: helm install graylog ./charts/graylog
• All pods reach Running state: kubectl rollout status statefulset/graylog
• Helm tests pass: helm test graylog

Functional (if applicable)

• Web UI accessible and login works
• DataNodes visible in System > Cluster Configuration
• Inputs can be created and receive data

Upgrade (if applicable)

• Upgrade from previous release succeeds
• Scaling up/down works correctly
• Configuration changes apply correctly

Specific to this PR

• helm unittest charts/graylog — 144 tests across 17 suites, all passing. New tests/affinity_test.yaml asserts each anti-affinity selector against the pod labels in the same rendered document, which is the check that catches a selector matching nothing. New tests/graylog_secret_test.yaml covers root-password rotation and the C-06 guard.
• Verified the MongoDB pod label on a live cluster (app: <crName>-svc, with none of the chart's labels present) — the anti-affinity selector was written against labels that do not exist on those pods.
• Root-password paths verified with helm upgrade --dry-run=server against a live release: upgrade with no explicit password preserves the stored hash; upgrade with graylog.config.rootPassword rotates it; the pepper and MongoDB URI are unchanged on both paths.
• Confirmed both root-password bugs reproduce on origin/main in a clean worktree, so they are pre-existing rather than introduced here.
• Not yet done: real (non-dry-run) install and upgrade, helm test, and UI login. The lookup-based restore branch cannot be reached from a unit test, so the live upgrade check is the only coverage for it and should be repeated before merge.

Notes for reviewers

• Verify all applicable tests above pass
• Validate that the linked issues are no longer reproducible, if applicable
• Sync up with the author before merging
• The commit history should be preserved - use rebase-merge or standard merge options when applicable

Behaviour changes for existing users — worth calling out in the release notes:

• PDBs are now created by default. Node drains will block rather than proceed when they would violate minAvailable; that is the intended protection, but it changes drain behaviour on small clusters.
• The MongoDB default goes from 2 data + 1 arbiter to 3 data members, which increases resource cost. Existing releases pin their own values, so this affects new installs and anyone relying on the chart default.
• graylog.config.rootPassword now takes effect on upgrade. Anyone who previously set it and saw no change should expect the password to actually rotate on their next upgrade.

[bemcculley-graylog]

LGTM!