feat: add etcd-based leader election to reduce kubeapi coupling

[eveningcafe]

Summary

This PR adds etcd-based leader election as an alternative backend for environments where Kubernetes API server stability is a concern (e.g., managed Kubernetes services).

Motivation

In managed Kubernetes environments, the kubeapi server can become unstable or slow under load. This causes issues with the default Kubernetes Lease-based leader election:

• Leader election failures when kubeapi is unavailable
• Unnecessary leader transitions during kubeapi instability
• Tight coupling between controller availability and kubeapi health

By using etcd directly for leader election:

• Reduced kubeapi coupling: Leader election operates independently of kubeapi availability
• Improved stability: Controller continues functioning even when kubeapi is slow/unstable
• Faster failover: Direct etcd access bypasses K8s API server overhead

Changes

• New package internal/leaderelection/etcd/ implementing resourcelock.Interface
• New CLI flag: --leader-election-backend (options: lease, etcd)
• Environment variables for etcd configuration:
  • ETCD_ENDPOINTS (required): comma-separated list of etcd endpoints
  • ETCD_CERT_FILE, ETCD_KEY_FILE: TLS client certificate
  • ETCD_CA_FILE: CA certificate for TLS verification
  • ETCD_USERNAME, ETCD_PASSWORD: authentication credentials

Usage

# Default (Kubernetes Lease API) - no changes needed
./kong-ingress-controller

# Using etcd backend
export ETCD_ENDPOINTS="http://etcd-0:2379,http://etcd-1:2379,http://etcd-2:2379"
./kong-ingress-controller --leader-election-backend=etcd

Kubernetes Deployment Example

containers:
- name: controller
  image: kong/kubernetes-ingress-controller:latest
  args:
  - --leader-election-backend=etcd
  env:
  - name: ETCD_ENDPOINTS
    value: "http://etcd-0.etcd:2379,http://etcd-1.etcd:2379,http://etcd-2.etcd:2379"

Test Plan

• Unit tests for config and resourcelock packages (22 test cases)
• Integration test with 3-node etcd cluster
• Tested leader failover (pod deletion → new leader elected)
• Tested etcd node failure → leader election continues
• Default lease backend continues to work unchanged

Checklist

• Code compiles without errors
• Follows existing code patterns (implements resourcelock.Interface)
• Unit tests added
• go vet and gofmt pass

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ eveningcafe
❌ Ngô Quang Hòa

---

Ngô Quang Hòa seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}