Make enabledIPv6 work without breaking single-stack clusters - #2310
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Improves IPv6 handling across aggregator listeners, metrics endpoints, and Kubernetes Services while preserving existing allocations.
Changes:
- Detects allocatable Service IP families and preserves assigned families during reconciliation.
- Adds configurable metrics bind addresses and corrects Fluent Bit/syslog-ng listeners.
- Fixes syslog-ng metrics overrides and config-reloader scraping.
Reviewed changes
Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pkg/sdk/logging/model/syslogng/config/source.go |
Adds source IP protocol rendering. |
pkg/sdk/logging/model/syslogng/config/config.go |
Configures the IPv6 syslog-ng listener. |
pkg/sdk/logging/model/syslogng/config/config_test.go |
Tests source listener rendering. |
pkg/sdk/logging/api/v1beta1/common_types.go |
Adds metrics binding and Service family helpers. |
pkg/sdk/logging/api/v1beta1/common_types_test.go |
Tests metrics bind resolution. |
pkg/resources/syslogng/syslogng.go |
Carries detected families into reconciliation. |
pkg/resources/syslogng/statefulset.go |
Corrects config-reloader metrics port. |
pkg/resources/syslogng/service.go |
Applies family detection and Service overrides. |
pkg/resources/syslogng/service_test.go |
Tests syslog-ng Service fixes. |
pkg/resources/ipfamily/detector.go |
Introduces Service IP-family detection. |
pkg/resources/ipfamily/detector_test.go |
Tests detection and retry behavior. |
pkg/resources/fluentd/service.go |
Preserves and applies Service families. |
pkg/resources/fluentd/service_test.go |
Tests Service upgrade behavior. |
pkg/resources/fluentd/fluentd.go |
Stores detected cluster families. |
pkg/resources/fluentd/configsecret.go |
Resolves the metrics bind address. |
pkg/resources/fluentd/config.go |
Renders the configured metrics bind. |
pkg/resources/fluentd/appconfigmap_test.go |
Updates reconciler construction. |
pkg/resources/fluentbit/service.go |
Applies detected Service families. |
pkg/resources/fluentbit/service_test.go |
Tests family-aware metrics Services. |
pkg/resources/fluentbit/fluentbit.go |
Stores detected cluster families. |
pkg/resources/fluentbit/configsecret.go |
Resolves the metrics bind address. |
pkg/resources/fluentbit/config.go |
Fixes Fluent Bit HTTP listener rendering. |
docs/configuration/crds/v1beta1/common_types.md |
Documents metrics.bind. |
controllers/logging/logging_controller.go |
Integrates family detection into reconcilers. |
config/crd/bases/logging.banzaicloud.io_syslogngconfigs.yaml |
Adds syslog-ng bind schema. |
config/crd/bases/logging.banzaicloud.io_loggings.yaml |
Adds Logging bind schemas. |
config/crd/bases/logging.banzaicloud.io_fluentdconfigs.yaml |
Adds Fluentd bind schema. |
config/crd/bases/logging.banzaicloud.io_fluentbitagents.yaml |
Adds Fluent Bit bind schema. |
CLAUDE.md |
Documents allocated Service field handling. |
charts/logging-operator/crds/logging.banzaicloud.io_syslogngconfigs.yaml |
Synchronizes syslog-ng chart CRD. |
charts/logging-operator/crds/logging.banzaicloud.io_loggings.yaml |
Synchronizes Logging chart CRD. |
charts/logging-operator/crds/logging.banzaicloud.io_fluentdconfigs.yaml |
Synchronizes Fluentd chart CRD. |
charts/logging-operator/crds/logging.banzaicloud.io_fluentbitagents.yaml |
Synchronizes Fluent Bit chart CRD. |
charts/logging-operator/charts/logging-operator-crds/templates/logging.banzaicloud.io_syslogngconfigs.yaml |
Updates packaged syslog-ng CRD template. |
charts/logging-operator/charts/logging-operator-crds/templates/logging.banzaicloud.io_loggings.yaml |
Updates packaged Logging CRD template. |
charts/logging-operator/charts/logging-operator-crds/templates/logging.banzaicloud.io_fluentdconfigs.yaml |
Updates packaged Fluentd CRD template. |
charts/logging-operator/charts/logging-operator-crds/templates/logging.banzaicloud.io_fluentbitagents.yaml |
Updates packaged Fluent Bit CRD template. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8 tasks
csatib02
force-pushed
the
fix/ipv6-service-families
branch
from
August 7, 2026 19:19
37d248c to
8d5f9ea
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 39 out of 39 changed files in this pull request and generated no new comments.
Suppressed comments (2)
pkg/resources/ipfamily/detector.go:63
nilmeans both “IPv4-only” and “the probe failed.” The syslog-ng config path converts it toClusterHasIPv6=false, while Service generation still setsPreferDualStackwithout pinning a family. On an IPv6-only cluster, a transient dry-run failure can therefore render an IPv4-onlynetwork()listener and then create an IPv6 Service, recreating the ingestion outage. Propagate the probe error (or model the unknown state separately) and abort/requeue instead of continuing withnil.
hasIPv6, err := d.allocatable(ctx, namespace, corev1.IPv6Protocol)
if err != nil {
return nil
}
hasIPv4, err := d.allocatable(ctx, namespace, corev1.IPv4Protocol)
if err != nil {
return nil
pkg/sdk/logging/api/v1beta1/common_types.go:78
Metricsis also used bySyslogNGSpec.Metrics, so this field is accepted in the generated syslog-ng CRDs, butsyslogNGMetricsSidecarContaineronly passes--socket.pathand never consumesBind/BindAddress. Consequentlyspec.metrics.bindis silently ignored for syslog-ng and the documented default is not applied. Wire the address into the exporter’s listen option, or avoid exposing this field for syslog-ng if the exporter cannot support it.
Kubernetes does not allow the primary IP family of an existing Service to change, and rejects the attempt as a validation error rather than an immutable-field one, so operator-tools cannot recognise it and the delete/recreate fallback never engages. Since 0321c1d asks for an IPv6-primary Service where 6.7.0 asked for IPv4-primary, upgrading a cluster running with enabledIPv6 left the fluentd and syslog-ng Services permanently failing to reconcile, which stalled the rest of the chain behind them. The before-update hook already pins clusterIP and clusterIPs to whatever the API server handed out; ipFamilies now follows the same rule. The policy is pinned only when it would narrow an existing multi-family Service, so enabling IPv6 on an existing Service still widens it to dual-stack. Only the two aggregator Services are non-headless and take this path. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
The failure mode is silent: operator-tools only recreates on messages that read as immutable-field errors, and neither an IP family flip on an existing Service nor a family the cluster has no range for produces one. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
EnableIPv6Options hard-coded ipFamilies to [IPv6, IPv4] whenever enabledIPv6 was set. The API server rejects a family it has no service range for, and it does so at create, where there is no recovery path at all: on a single-stack IPv4 cluster the Service is never allocated and the reconcile stalls behind it. 2777a11 extended that reach to the Fluent Bit metrics Services, which worked at 6.7.0. The cluster is now asked directly, by dry-run creating a probe Service pinned to IPv6. A dry-run leaves nothing behind, needs no RBAC beyond the services create the operator already holds, and has no version floor. An IPv4 control probe separates a missing IPv6 range from an admission webhook that rejects every Service, and an unresolved answer is retried rather than cached, so a cluster that gains an IPv6 range is picked up without a restart. Where the cluster has IPv6 the primary family is unchanged. Where it does not, the families are left to the cluster and PreferDualStack still applies. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
syslogNG.enabledIPv6 only ever changed the Services. syslog-ng itself has no ip() on its network source, so it keeps the upstream default of ip(0.0.0.0) ip-protocol(4) and listens on IPv4 alone. Once 0321c1d made the shared helper ask for an IPv6 primary, the port 601 Service resolved to an AAAA that Fluent Bit's forward output prefers, and kube-proxy sent the connection to a pod address with no listener behind it. That took ingestion down, not just metrics. The source now binds "::" with ip-protocol(6) under the same flag, which still accepts IPv4-mapped clients wherever bindv6only is off, so the two sides of the flag finally agree. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
The single-stack branch emitted "Listen", which is not a [SERVICE] key, so Fluent Bit ignored it and fell back to its own default. The default happened to be the same address, which is why nothing looked wrong. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
metricsService and bufferVolumeMetricsService have been on the CRD since syslog-ng support landed, but nothing ever read them, so whatever a user put there was dropped without a word. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
enabledIPv6 was the only way to move the metrics listener, and it moves it to [::], which fluent-plugin-prometheus serves through WEBrick with IPV6_V6ONLY set. Reaching that listener then needs an IPv6 primary on the Service, which is what tied metrics scraping to the cluster's IP families in the first place. A bind address on the Metrics type breaks that coupling: enabledIPv6 now only picks the default, and a user who needs a specific address names it. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
The probe asked whether IPv6 was allocatable and nothing else, while the Service always named IPv4 alongside it. On an IPv6-only cluster the probe succeeded and every Service then carried an IPv4 entry the API server had no range for, so it was rejected at create exactly as an IPv6 entry is on an IPv4-only cluster. That is the failure this package exists to prevent, left open in the mirror direction. The detector now reports the allocatable families and the Service names those. A plain IPv4 cluster still names nothing, which is what it did before. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
Pinning the allocated families overrides a single-stack policy the user asked for through serviceOverrides, so a deliberate downgrade to one family never happened and nothing said so. The reconcilers now log it. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
Probing each family on its own merit lost the control the previous shape had. A cluster serves at least one family, so two rejections mean something refuses every Service, not that the cluster has no ranges. The detector treated that as a resolved answer of "no families" and kept it for the life of the process, so an admission webhook that was later fixed left the operator on the wrong answer until a restart. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
… metrics bind Three points from review. The syslog-ng source bound "::" whenever enabledIPv6 was set. That reaches IPv4 clients only through v4-mapped addresses, which net.ipv6.bindv6only turns off. On a node with that setting and a cluster without IPv6, the Service is IPv4 and the source had no IPv4 listener, so ingestion failed. The bind now follows the same cluster answer the Services use. The buffer-metrics sidecars hard-coded their listen address, so a bind set on bufferVolumeMetrics was accepted and dropped. They pass it through now. An unset bind still yields the wildcard address. A cluster with no IPv6 range is re-probed instead of cached, so a range added to a running cluster is picked up without restarting the operator. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
…ermined Families returned nil both for a plain IPv4 cluster and for a probe that could not run. The syslog-ng config read that as "no IPv6" and rendered an IPv4 listener, while Service generation kept PreferDualStack and let the API server pick. On an IPv6-only cluster a single failed probe therefore produced an IPv6 Service in front of an IPv4 listener, which is the outage this branch exists to remove. The probe now returns an error and the reconcile stops, so the Service and the listener are always built from the same answer. Also drops the config-reloader port test, which moved to another pull request with the change it covered. Signed-off-by: Bence Csati <bence.csati@axoflow.com>
…ress Signed-off-by: Bence Csati <bence.csati@axoflow.com>
OverOrion
approved these changes
Aug 10, 2026
csatib02
force-pushed
the
fix/ipv6-service-families
branch
from
August 10, 2026 14:33
7c9cd10 to
03f0a28
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
enabledIPv6produced Services the cluster could not accept, and listeners that did notmatch those Services. Three regressions landed after 6.7.0, all from one shared helper.
0321c1d2changed the requested primary IP family from IPv4 to IPv6. Kubernetes does notallow the primary family of an existing Service to change. It reports the refusal as
invalid, not as immutable, so operator-tools does not recognise it and never recreates the
Service. Upgrading a cluster with
enabledIPv6left the fluentd or syslog-ng Servicefailing to reconcile forever, which stalled every reconciler behind it.
The same helper also names a family outright. A cluster with no range for that family
rejects the Service at create, where there is no recovery path. This branch asks the API
server which families it can allocate, with a dry-run Service, and names only those. The
probe needs no new RBAC and has no version floor.
syslog-ng was affected differently. Its
network()source never setip(), so it listenedon IPv4 only while its Service became IPv6-primary. Fluent Bit resolved the Service to an
AAAA record and the connection reached a pod address with no listener. That took ingestion
down, not just metrics.
The last commits remove the cause. A
bindaddress on theMetricstype decouples themetrics endpoint from the Service IP families, so wanting IPv6 metrics no longer forces an
IPv6-primary Service. Three smaller defects found on the way are fixed too: the Fluent Bit
config used an invalid
Listenkey, the syslog-ng config-reloader listened on a portnothing scraped, and the syslog-ng metrics service overrides were never read.
Context: #2100, #2238, #1835 and #2090 are all closed. #2100 was closed by the commit that
caused two of the regressions above.
Test plan
Verified on a KIND cluster (v1.36.1, service CIDR
10.96.0.0/16) and on envtest APIservers driving the real reconcilers.
make lint && make testfluentd.enabledIPv6: true. TheService reconciles and keeps its allocated addresses.
enabledIPv6: true. Services are createdand the Logging converges. The families the previous code asked for are rejected with
spec.ipFamilies[0]: Invalid value: "IPv6": not configured on this cluster.enabledIPv6: true. Services are created.syslogNG.enabledIPv6: true. A message reaches theip("::") ip-protocol(6)listener through an IPv4 Service.metrics.bindand confirm it reaches the rendered aggregator config.Not covered: there is no e2e suite for
enabledIPv6, so the dual-stack IPv6-primaryingestion path is verified only at the layers below a cluster. The bind was run against
ghcr.io/axoflow/axosyslog:4.25.0and the Services against envtest.