feat: add diagnostic hint log when driver cannot reach kube-apiserver on startup - #2055
feat: add diagnostic hint log when driver cannot reach kube-apiserver on startup#2055pujitha24 wants to merge 1 commit into
Conversation
… on startup
/kind feature
**What this PR does / why we need it**:
Motivation:
When the driver fails to start because it cannot reach the kube-apiserver
(e.g. RESTMapper/discovery client construction in ctrl.NewManager, or cache
sync in mgr.Start failing due to DNS, connection refused, or timeouts), the
existing logs ("failed to start manager", "failed to run manager") give no
hint about the likely root cause. Issue kubernetes-sigs#1753 reports exactly this: a driver
pod failed to initialize with only a generic wrapped error, with no guidance
that network connectivity to the kube-apiserver should be checked.
Approach:
Add an `isNetworkError` helper in cmd/secrets-store-csi-driver/main.go that
classifies an error as network-related if it unwraps to a `net.Error` (covers
`*net.DNSError`, `*net.OpError`, etc.) or its message contains one of a small
set of well-known connectivity failure substrings ("connection refused", "no
such host", "i/o timeout", "context deadline exceeded"). When this helper
returns true at the two manager-startup failure points, an additional
klog.ErrorS call logs: "the driver is unable to communicate with the
kube-apiserver, check network connectivity between the node and the
kube-apiserver". This mirrors the existing isMaxRecvMsgSizeError pattern in
pkg/secrets-store/provider_client.go.
This change is purely additive: it does not alter control flow, return
values, exit codes, or any existing log message. The original error is still
logged and returned/panicked exactly as before; only an extra diagnostic log
line is emitted when the error looks network-related. User-visible behavior
(pod restarts on manager start failure) is unchanged before/after this fix —
the benefit is a clearer, more actionable log line for operators debugging a
non-starting driver pod, not a change in recovery behavior.
Validation:
- `go build ./...` passes.
- `go test ./cmd/... -v` passes, including the new table-driven
TestIsNetworkError covering nil errors, unrelated errors, wrapped
net.DNSError, and each of the matched substring cases.
- `gofmt -l` and `go vet ./cmd/...` report no issues.
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
|
This issue is currently awaiting triage. If secrets-store-csi-driver contributors determine this is a relevant issue, they will accept it by applying the The DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Welcome @pujitha24! |
|
Hi @pujitha24. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: pujitha24 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Pull request overview
This PR improves startup troubleshooting by emitting an additional, actionable log hint when the driver appears unable to communicate with the kube-apiserver during manager initialization or startup, helping operators diagnose common node→apiserver connectivity failures.
Changes:
- Added
isNetworkErrorincmd/secrets-store-csi-driver/main.goto classify likely kube-apiserver connectivity-related startup errors. - Emitted an extra
klog.ErrorShint at both manager construction and manager start failure points whenisNetworkErrormatches. - Added a table-driven unit test for
isNetworkError.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| cmd/secrets-store-csi-driver/main.go | Adds network-error classification and an extra diagnostic hint log on manager startup failures. |
| cmd/secrets-store-csi-driver/main_test.go | Adds unit tests covering isNetworkError classification behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if isNetworkError(err) { | ||
| klog.ErrorS(err, "the driver is unable to communicate with the kube-apiserver, check network connectivity between the node and the kube-apiserver") | ||
| } |
e3b5f96 to
b543944
Compare
/kind feature
What this PR does / why we need it:
Motivation:
When the driver fails to start because it cannot reach the kube-apiserver
(e.g. RESTMapper/discovery client construction in ctrl.NewManager, or cache
sync in mgr.Start failing due to DNS, connection refused, or timeouts), the
existing logs ("failed to start manager", "failed to run manager") give no
hint about the likely root cause. Issue #1753 reports exactly this: a driver
pod failed to initialize with only a generic wrapped error, with no guidance
that network connectivity to the kube-apiserver should be checked.
Approach:
Add an
isNetworkErrorhelper in cmd/secrets-store-csi-driver/main.go thatclassifies an error as network-related if it unwraps to a
net.Error(covers*net.DNSError,*net.OpError, etc.) or its message contains one of a smallset of well-known connectivity failure substrings ("connection refused", "no
such host", "i/o timeout", "context deadline exceeded"). When this helper
returns true at the two manager-startup failure points, an additional
klog.ErrorS call logs: "the driver is unable to communicate with the
kube-apiserver, check network connectivity between the node and the
kube-apiserver". This mirrors the existing isMaxRecvMsgSizeError pattern in
pkg/secrets-store/provider_client.go.
This change is purely additive: it does not alter control flow, return
values, exit codes, or any existing log message. The original error is still
logged and returned/panicked exactly as before; only an extra diagnostic log
line is emitted when the error looks network-related. User-visible behavior
(pod restarts on manager start failure) is unchanged before/after this fix —
the benefit is a clearer, more actionable log line for operators debugging a
non-starting driver pod, not a change in recovery behavior.
Validation:
go build ./...passes.go test ./cmd/... -vpasses, including the new table-drivenTestIsNetworkError covering nil errors, unrelated errors, wrapped
net.DNSError, and each of the matched substring cases.
gofmt -landgo vet ./cmd/...report no issues.Fixes #1753
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com