WIP: Add support for zone-redundant load balancers #5944
+1,483
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k8s-ci-robot
added
release-note
k8s-ci-robot
added
the
cncf-cla: yes
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
size/XL
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
7572b39 to
67685f0
Compare
k8s-ci-robot
added
needs-rebase
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
67685f0 to
2e5d373
Compare
k8s-ci-robot
added
size/XL
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
2e5d373 to
e29bc2e
Compare
k8s-ci-robot
removed
the
do-not-merge/invalid-commit-message
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5944 +/- ##
==========================================
+ Coverage 44.54% 44.68% +0.14%
==========================================
Files 279 279
Lines 25140 25288 +148
==========================================
+ Hits 11199 11301 +102
- Misses 13128 13160 +32
- Partials 813 827 +14 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
|
@bryan-cox can you add this new functionality to the existing E2E scenario for a private cluster, which ships with/ an internal LB? E.g.: $ git diff templates/flavors/private/patches/private-lb.yaml
diff --git a/templates/flavors/private/patches/private-lb.yaml b/templates/flavors/private/patches/private-lb.yaml
index 76e1539df..a2933e299 100644
--- a/templates/flavors/private/patches/private-lb.yaml
+++ b/templates/flavors/private/patches/private-lb.yaml
@@ -7,6 +7,10 @@ spec:
apiServerLB:
name: ${CLUSTER_NAME}-internal-lb
type: Internal
+ availabilityZones:
+ - "1"
+ - "2"
+ - "3"
nodeOutboundLB:
frontendIPsCount: 1
controlPlaneOutboundLB:After you apply the above changes to the template partial above, render updated templates w/ |
Contributor
|
/test pull-cluster-api-provider-azure-e2e-optional |
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
e29bc2e to
3b77777
Compare
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
Contributor
Author
|
/retest |
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
3b77777 to
6fa7de7
Compare
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
36f5c8f to
e69a17e
Compare
Contributor
Author
|
Attempting to get the PR back to its stable state before attempting to address #5944 (comment) again. |
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
k8s-ci-robot
added
size/XXL
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
461ebb0 to
9ca4c6c
Compare
k8s-ci-robot
added
size/XL
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
a37ab02 to
b5bbbad
Compare
k8s-ci-robot
added
size/XXL
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
Add comprehensive documentation for zone-redundant load balancer feature: - Explain Azure zone redundancy concepts for load balancers - Provide configuration examples for all load balancer types: - Internal load balancers (API server) - Public load balancers - Node outbound load balancers - Control plane outbound load balancers - Include complete highly available cluster example - Document important considerations: - Immutability of zones after creation - Region support requirements - Standard SKU requirement - Backend pool placement best practices - Provide migration guidance for existing clusters - Add troubleshooting section - Document best practices
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
b5bbbad to
aee70a4
Compare
k8s-ci-robot
added
the
do-not-merge/invalid-commit-message
- Add AvailabilityZones field to LoadBalancerSpec API - Implement zone support in service layer for frontend IP configs - Add webhook validation for zone immutability - Update generated CRD manifests - Add zone redundancy to private cluster flavor
- Add unit tests for zone configuration on frontend IPs - Add E2E test for zone-redundant LB verification - Add apiserver-ilb-zones flavor for E2E testing
bryan-cox
force-pushed
the
issue-5709-lb-zone-redundancy
branch
from
aee70a4 to
daacbf5
Compare
k8s-ci-robot
removed
the
do-not-merge/invalid-commit-message
Contributor
Author
|
/test pull-cluster-api-provider-azure-e2e-optional |
…or public LBs Azure does not allow zones on frontend IP configurations that reference public IP addresses. For public load balancers, zone-redundancy is achieved by setting zones on the public IP address resource itself. This commit: - Modifies getFrontendIPConfigs() to only apply zones for internal LBs - Updates PublicIPSpecs() to use LB availability zones for public IPs - Adds getPublicIPFailureDomains() helper function - Updates E2E test to verify zones on frontend (internal) vs public IPs (public) - Updates test template to use internal API server LB with outbound LBs - Adds documentation explaining Azure's zone behavior with doc references Azure error avoided: LoadBalancerFrontendIPConfigCannotHaveZoneWhenReferencingPublicIPAddress Reference: https://learn.microsoft.com/en-us/azure/reliability/reliability-load-balancer#zone-redundant-load-balancer Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix godot lint error by simplifying comment format in cluster.go - Update kustomization patch to properly configure zone-redundant LBs: - Internal apiServerLB with zones on frontend IP configuration - Control plane outbound LB with zones (propagate to public IPs) - Node outbound LB with zones (propagate to public IPs) - Regenerate template from kustomize source The template was previously edited directly but is auto-generated by kustomize from the patch directory. This commit updates the source patch file and regenerates the template properly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
bryan-cox
changed the title
k8s-ci-robot
added
the
do-not-merge/work-in-progress
Contributor
|
@bryan-cox: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Contributor
Author
|
/test pull-cluster-api-provider-azure-apiversion-upgrade |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it:
This PR implements support for configuring availability zones on Azure load balancers to enable zone-redundant configurations for high availability.
Azure load balancers can be configured as zone-redundant to ensure high availability across multiple availability zones within a region. This feature allows users to specify availability zones (1, 2, 3) on load balancers, which are then set on the frontend IP configurations.
Key changes:
AvailabilityZonesfield toLoadBalancerSpecAPIWhich issue(s) this PR fixes:
Fixes #5709
Special notes for your reviewer:
This implementation follows Azure's zone redundancy model:
The E2E test is optional and creates a cluster with zone-redundant load balancers to verify the feature works end-to-end in Azure.
TODOs:
Release note: