Skip to content

EAS: add Extension API Server with multi-active HA#1388

Open
zhengxiexie wants to merge 1 commit intovmware-tanzu:mainfrom
zhengxiexie:topic/zhengxie/main/display_ip
Open

EAS: add Extension API Server with multi-active HA#1388
zhengxiexie wants to merge 1 commit intovmware-tanzu:mainfrom
zhengxiexie:topic/zhengxie/main/display_ip

Conversation

@zhengxiexie
Copy link
Contributor

@zhengxiexie zhengxiexie commented Mar 10, 2026

✨ What's Changed

NSX Extension API Server (EAS)

New sidecar in the nsx-ncp pod exposing NSX IP usage data via Kubernetes aggregated API (eas.nsx.vmware.com/v1alpha1).

Resources:

  • VPCIPAddressUsage — per-VPC IP block allocation percentages
  • IPBlockUsage — IP block used/available ranges
  • SubnetIPPools — subnet IP pool details (excludes DHCP subnets)
  • SubnetDHCPServerConfigStats — DHCP server config and pool stats

All resources support JSON and kubectl Table output (kubectl get ... -o wide).

Server Architecture (pkg/eas/server/)

  • Generic map-based resource dispatch via registerResource[Item, List]() — no per-resource switch/case
  • controller-runtime Runnable lifecycle (context-aware shutdown, manager-managed)
  • HTTP method gating: only GET/HEAD allowed; others return 405
  • Kubernetes StatusError responses for consistent error format
  • Multi-active mode: all replicas serve concurrently — no leader election needed since EAS is read-only. K8s Service load-balances across Ready pods automatically.

File Structure

cmd_eas/main.go              # Thin bootstrap (~90 lines)
pkg/eas/server/
  server.go                  # EASServer struct, Runnable lifecycle, HTTP listener
  handler.go                 # Generic resource dispatch, route registration, path parsing
  response.go                # JSON/Table/Error response helpers
  table.go                   # Column definitions + summary formatters
  server_test.go             # Listener config tests
  handler_test.go            # Method gating, path parsing, dispatch tests
  response_test.go           # Error encoding, table metadata tests
pkg/eas/
  vpcprovider.go             # Namespace → VPC resolution via K8s CRDs
  storage/                   # Per-resource NSX data fetching
  converter/                 # NSX model → EAS API type conversion
pkg/apis/eas/v1alpha1/       # API types + deepcopy
pkg/util/path.go             # Shared path segment utilities

🎯 Design Decisions

Why no leader election? Unlike nsx-operator which writes to NSX, EAS only reads. Multiple replicas returning the same data is harmless and enables:

  • Zero-downtime failover (no 30s lease expiry + transition delay)
  • Load balancing across replicas via K8s Service
  • Simpler code (no lease management, pod labeling, transition waits)

✅ Testing

Unit Tests

$ go test ./pkg/eas/server/...
ok   github.com/vmware-tanzu/nsx-operator/pkg/eas/server   0.9s  (9 tests)

$ go test ./pkg/util -run '^(TestSplitPathSegments|TestLastPathSegment)$'
ok   github.com/vmware-tanzu/nsx-operator/pkg/util          0.3s

E2E Verification (Testbed 15412)

Deployed to Supervisor cluster (10.161.114.97, NSX 10.161.117.160):

# API JSON Table Verification
1 VPCIPAddressUsage VPC name, 3 ipBlocks, CIDR/percentage match NSX
2 IPBlockUsage 3 records, name prefix correct, ranges match
3 SubnetIPPools DHCP_SERVER subnet correctly filtered
4 SubnetDHCPServerConfigStats dhcpServerId/poolSize match NSX

Multi-active verified: EAS starts serving immediately after NSX client init (no election delay). Health check and aggregated API (kubectl get vpcipaddressusages -A) confirmed working.

🔄 Backward Compatibility

No API changes to existing resources. EAS is an additive feature — new sidecar container, new APIService, new CRDs.

@zhengxiexie zhengxiexie changed the title EAS: refactor server into pkg/eas/server with lifecycle, dispatch, and hardening [WIP]EAS: refactor server into pkg/eas/server with lifecycle, dispatch, and hardening Mar 10, 2026
@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/display_ip branch 2 times, most recently from 0fa2851 to dd97093 Compare March 10, 2026 07:09
@codecov-commenter
Copy link

codecov-commenter commented Mar 10, 2026

Codecov Report

❌ Patch coverage is 12.10407% with 777 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.17%. Comparing base (b153b78) to head (bc806a7).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pkg/eas/converter/converter.go 0.00% 142 Missing ⚠️
pkg/eas/server/handler.go 20.48% 126 Missing and 6 partials ⚠️
pkg/util/cert.go 0.00% 132 Missing ⚠️
pkg/eas/storage/subnetippools.go 0.00% 63 Missing ⚠️
cmd_eas/main.go 0.00% 60 Missing ⚠️
pkg/eas/vpcprovider.go 0.00% 59 Missing ⚠️
pkg/eas/server/server.go 18.96% 47 Missing ⚠️
pkg/eas/storage/subnetdhcpstats.go 0.00% 42 Missing ⚠️
pkg/eas/server/table.go 0.00% 31 Missing ⚠️
pkg/eas/storage/vpcipaddressusage.go 0.00% 31 Missing ⚠️
... and 2 more

❌ Your patch status has failed because the patch coverage (12.10%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1388      +/-   ##
==========================================
- Coverage   76.74%   74.17%   -2.57%     
==========================================
  Files         151      163      +12     
  Lines       21162    22191    +1029     
==========================================
+ Hits        16240    16460     +220     
- Misses       3762     4554     +792     
- Partials     1160     1177      +17     
Flag Coverage Δ
unit-tests 74.17% <12.10%> (-2.57%) ⬇️
Files with missing lines Coverage Δ
pkg/config/config.go 71.22% <ø> (ø)
pkg/nsx/client.go 95.00% <100.00%> (+0.09%) ⬆️
pkg/util/path.go 100.00% <100.00%> (ø)
pkg/eas/server/response.go 83.92% <83.92%> (ø)
pkg/eas/storage/ipblockusage.go 0.00% <0.00%> (ø)
pkg/eas/server/table.go 0.00% <0.00%> (ø)
pkg/eas/storage/vpcipaddressusage.go 0.00% <0.00%> (ø)
pkg/eas/storage/subnetdhcpstats.go 0.00% <0.00%> (ø)
pkg/eas/server/server.go 18.96% <18.96%> (ø)
pkg/eas/vpcprovider.go 0.00% <0.00%> (ø)
... and 5 more

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zhengxiexie zhengxiexie changed the title [WIP]EAS: refactor server into pkg/eas/server with lifecycle, dispatch, and hardening EAS: add Extension API Server with multi-active HA Mar 12, 2026
@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/display_ip branch from dd97093 to 642da0f Compare March 12, 2026 07:19
Add the Extension API Server (EAS) as an aggregated API that exposes
NSX IP usage data (VPC IP address usage, IP block usage, subnet IP
pools, subnet DHCP stats) via standard Kubernetes API endpoints.

EAS is read-only and runs in multi-active mode — all replicas serve
concurrently without leader election, with the Kubernetes Service
providing automatic load balancing.

TLS certificates are generated automatically: a self-signed CA and
server certificate are created, stored in a Kubernetes Secret, and
injected into the APIService caBundle. Certificates refresh every
30 days.

Co-Authored-By: Oz <oz-agent@warp.dev>
Change-Id: I967903412334a30e31f815d0ed4fdbf53afd60f6
@zhengxiexie zhengxiexie force-pushed the topic/zhengxie/main/display_ip branch from 805b822 to bc806a7 Compare March 12, 2026 07:52
@zhengxiexie
Copy link
Contributor Author

/e2e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants