Skip to content

portforward: Auto-reconnect service-backed port-forwards on pod restart - #7273

Open
Slambot01 wants to merge 1 commit into
kubernetes-sigs:mainfrom
Slambot01:feat/portforward-auto-reconnect
Open

portforward: Auto-reconnect service-backed port-forwards on pod restart#7273
Slambot01 wants to merge 1 commit into
kubernetes-sigs:mainfrom
Slambot01:feat/portforward-auto-reconnect

Conversation

@Slambot01

Copy link
Copy Markdown
Contributor

Summary

This PR adds automatic reconnection for service-backed port-forwards by resolving a new pod via the Kubernetes Endpoints API when the target pod is deleted or restarted (e.g. during a rolling update). Previously, the port-forward was marked as Stopped and required manual restart.

Related Issue

Fixes #7272

Changes

  • Added RECONNECTING status constant and rConf field to portForward struct in backend/pkg/portforward/handler.go
  • Added resolveServicePod() to query the Endpoints API for a ready pod backing a service
  • Added attemptReconnect() with exponential backoff (3 attempts: 5s → 10s → 20s)
  • Updated monitorPodAndManagePortForward() to attempt auto-reconnect for service-backed port-forwards instead of immediately stopping
  • Added reconnect_test.go with 5 unit tests for resolveServicePod using fake clientsets
  • Added PORT_FORWARD_RECONNECTING_STATUS constant and reconnecting UI (spinner + "Reconnecting..." text) in frontend/src/components/common/Resource/PortForward.tsx
  • Updated frontend/src/components/portforward/index.tsx to show a warning-colored status label for reconnecting port-forwards

Steps to Test

  1. Start Headlamp locally against a cluster (minikube/kind)
  2. Create a Deployment + Service: kubectl create deployment nginx --image=nginx && kubectl expose deployment nginx --port=80
  3. Navigate to the Service in Headlamp and start a port-forward
  4. Delete the pod: kubectl delete pod -l app=nginx
  5. Observe: status briefly shows "Reconnecting" then auto-reconnects to the new pod
  6. Verify the same local port is still accessible
  7. Edge case: delete the Service itself → verify port-forward stops after 3 failed reconnect attempts

Notes for the Reviewer

  • Auto-reconnect only applies to Service-based port-forwards. Pod-only port-forwards retain existing stop-on-failure behavior since there is no way to discover a replacement pod without a Service selector.
  • resolveServicePod accepts kubernetes.Interface instead of *kubernetes.Clientset to enable testing with fake clientsets. All callers pass *kubernetes.Clientset which satisfies the interface.
  • The new rConf field is tagged json:"-" so it is excluded from all API responses.

@kubernetes-prow kubernetes-prow Bot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 15, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Slambot01
Once this PR has been reviewed and has the lgtm label, please assign joaquimrocha for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 15, 2026
@illume
illume requested a balanced review from Copilot August 16, 2026 06:32

@illume illume left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution.

The GitHub CI test job has snapshot failures. Run cd frontend && npm run test -- -u to regenerate the snapshots.

How to update snapshots

Run cd frontend && npm run test -- -u to regenerate all snapshots. Review the diff to make sure the visual changes are intentional, then commit the updated snapshot files.

The backend test job in CI is failing. Run cd backend && go test ./... to reproduce the errors locally.

How to run the backend tests

Run cd backend && go test ./... to see all failures. Fix the failing tests and commit the result.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds automatic recovery for service-backed port-forwards when their target pod restarts.

Changes:

  • Resolves replacement pods and retries forwarding with backoff.
  • Adds backend resolver tests.
  • Displays reconnecting states in the frontend.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.

File Description
backend/pkg/portforward/handler.go Implements reconnection lifecycle and status handling.
backend/pkg/portforward/reconnect_test.go Tests service endpoint resolution.
frontend/src/components/common/Resource/PortForward.tsx Adds reconnecting status and spinner.
frontend/src/components/portforward/index.tsx Adds reconnecting list status.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread backend/pkg/portforward/handler.go Outdated
Comment thread backend/pkg/portforward/handler.go Outdated
Comment thread backend/pkg/portforward/handler.go Outdated
Comment thread backend/pkg/portforward/handler.go Outdated
Comment thread backend/pkg/portforward/reconnect_test.go Outdated
Comment thread backend/pkg/portforward/handler.go Outdated
Comment thread frontend/src/components/common/Resource/PortForward.tsx
Comment thread frontend/src/components/portforward/index.tsx
Comment thread backend/pkg/portforward/handler.go
@Slambot01
Slambot01 requested a review from illume August 16, 2026 07:59
@illume
illume requested a balanced review from Copilot August 16, 2026 11:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 1 comment.

Suppressed comments (4)

backend/pkg/portforward/handler.go:717

  • Capturing the old channel prevents this goroutine from closing the replacement channel, but it still mutates the shared pfDetails below. If the old ForwardPorts exits after a replacement has become Running, either its error path or normal-exit path can mark the replacement Stopped and overwrite the cache. Associate status updates with a forwarder generation (or verify channel ownership under mu) before allowing this goroutine to mutate shared state.
	// Capture closeChan now so the deferred close targets THIS forwarder's
	// channel even if pfDetails.closeChan is swapped during reconnect.
	closeOnExit := pfDetails.closeChan

backend/pkg/portforward/handler.go:525

  • The added tests cover only EndpointSlice resolution; none executes this retry/cancellation lifecycle, which allowed the already-closed channel above to make every retry exit immediately. Add deterministic coverage for a successful retry, exhaustion, readiness failure followed by another attempt, and user cancellation, using injectable backoffs/forwarder setup so the tests remain fast.
		select {
		case <-time.After(delay):
		case <-pfDetails.closeChan:

frontend/src/components/common/Resource/PortForward.tsx:492

  • This reconnecting branch removes the Stop action from the Service/Pod detail view for the entire retry window, even though the port-forward list deliberately treats Reconnecting like Running and retains Stop. Keep the stop button available beside the spinner so users can cancel a reconnect without navigating away.
          ) : portForward.status === PORT_FORWARD_RECONNECTING_STATUS ? (
            <Box display={'flex'} alignItems="center" gap={1}>
              <CircularProgress size={16} />

backend/pkg/portforward/handler.go:506

  • The commit sequence needs cleanup: 9279bf8 broadly rewrites/fixes the feature introduced by 84cd8f5, and 108b0b0 then fixes lint/i18n fallout from that corrective commit. Please interactively rebase and squash/reorder these corrections into coherent feature commits so the reviewed history does not preserve intermediate broken states.
// attemptReconnect tries to re-establish a port-forward by finding a new pod
// behind the same service. It retries up to 3 times with exponential backoff
// (5s, 10s, 20s). Returns true if reconnection succeeded.
func attemptReconnect(

Comment thread backend/pkg/portforward/handler.go
@kubernetes-prow kubernetes-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 17, 2026
@Slambot01
Slambot01 force-pushed the feat/portforward-auto-reconnect branch from 0a13856 to a84695c Compare August 17, 2026 08:10
@kubernetes-prow kubernetes-prow Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 17, 2026
@Slambot01
Slambot01 force-pushed the feat/portforward-auto-reconnect branch from a84695c to 2c0182f Compare August 17, 2026 09:14
@kubernetes-prow kubernetes-prow Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 17, 2026
Squashes previous commits to fix i18n formatting diff issues
@Slambot01
Slambot01 force-pushed the feat/portforward-auto-reconnect branch from 2c0182f to b2fc735 Compare August 17, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port-forward auto-reconnect on pod restart for service-backed port-forwards

3 participants