fix: detect and remount stale NFS mounts in NodePublishVolume

[andyzhangx]

What type of PR is this?
/kind bug

What this PR does / why we need it:
When an NFS server restarts, existing mounts on nodes become stale. If a pod with fsGroup is then restarted, kubelet's applyFSGroup calls lstat on the mount path and fails with ESTALE (stale NFS file handle) before the CSI driver gets a chance to remount.

This PR fixes NodePublishVolume to detect stale mounts and automatically unmount + remount them, so that kubelet's subsequent applyFSGroup succeeds.

How does it work:

1. When the target path is already mounted, do an os.Lstat check
2. If ESTALE is detected, unmount the stale mount
3. Fall through to the normal mount path for a fresh mount

Which issue(s) this PR fixes:
Ref #927

Does this PR introduce a user-facing change?

Fix stale NFS mount detection in NodePublishVolume: pods with fsGroup can now recover after NFS server restart without manual intervention.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [andyzhangx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Copilot]

Pull request overview

Fixes kubelet applyFSGroup failures after NFS server restarts by detecting stale NFS mounts in NodePublishVolume and forcing an unmount/remount when a stale file handle is detected.

Changes:

• Add an os.Lstat check on already-mounted target paths to detect stale NFS handles.
• On ESTALE, unmount the target and proceed with a fresh mount.
• Introduce a helper (isStaleFileHandle) to classify stale-handle errors.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[andyzhangx]

Addressed the review comment — added unit tests for the stale mount detection path in 3ce23a8:

• Introduced lstatFunc package-level variable (defaults to os.Lstat) so tests can inject errors
• Added TestNodePublishVolume case that injects syscall.ESTALE, verifies unmount+remount succeeds
• Added TestIsStaleFileHandle covering ESTALE errno, wrapped ESTALE, string-match variants, and negative cases

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

pkg/nfs/nodeserver_test.go:33

• This test now imports syscall and uses syscall.ESTALE/ENOENT. Those constants are not available on Windows, so pkg/nfs unit tests may stop compiling on GOOS=windows. If Windows builds are still supported, gate these tests behind build tags (e.g. !windows) or avoid OS-specific errno constants in the test inputs.

import (
	"context"
	"errors"
	"fmt"
	"os"
	"reflect"
	"strings"
	"syscall"
	"testing"

	"github.com/container-storage-interface/spec/lib/go/csi"
	"github.com/kubernetes-csi/csi-driver-nfs/test/utils/testutil"
	"github.com/stretchr/testify/assert"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[andyzhangx]

/cherrypick release-4.13

[k8s-infra-cherrypick-robot]

@andyzhangx: new pull request created: #1109

Details

In response to this:

/cherrypick release-4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.