[ci-maintainer] Go Tests: widespread failures across pkg/agent, pkg/api, pkg/api/handlers, pkg/mcp, pkg/settings

[kubestellar-hive]

CI Issue

The Go Tests workflow (go test ./...) has widespread failures across many packages on main. These are persistent across multiple consecutive runs.

Evidence

Workflow: Go Tests (workflow 261783871)
Recent failing runs: 27451940255 (2026-06-13T01:07), 27450088228 (2026-06-13T00:02), 27444711112 (2026-06-12T21:42)

Failing packages

Package	Failure Type	Example
pkg/agent	Multiple test assertion failures	TestCodexProvider_StreamChatDrainsStderr, TestGitopsHandlers, TestServer_HandleHelmRollback
pkg/api	Test assertion failure	TestNormalizeKCAgentBaseURL_RejectsMissingHost
pkg/api/handlers	nil pointer panic	WebhookHandlers.ListWebhooks → k8s.MultiClusterClient(nil).DeduplicatedClusters
pkg/api/handlers/compliance	Assertion failure	TestWebhookDestination_SendPOSTsExpectedPayload
pkg/api/handlers/feedback	Assertion failure	TestRunAsyncGitHubOp_MultipleOperations
pkg/api/handlers/github	Assertion failure	TestNightlyE2EHandler_GetRunLogs_Success
pkg/api/handlers/mcp	Mock panic	Unexpected Method Call
pkg/api/handlers/missions	Timeout	TestMissions_BrowseConsoleKB_EmbeddedFallback
pkg/api/handlers/stellar	Assertion failure	TestRequireUser
pkg/api/middleware	Assertion failure	TestParseJWT_InvalidClaims
pkg/kagentiprovider	Assertion failure	TestDiscover
pkg/mcp	Timeout (90-120s each)	TestBridge_GetPods, TestBridge_FindPodIssues, TestBridge_GetEvents
pkg/settings	Assertion failure	TestValidation_SaveAll_EmptySettings

Critical: nil pointer panic in pkg/api/handlers

panic: runtime error: invalid memory address or nil pointer dereference
pkg/k8s.(*MultiClusterClient).ListClusters(0x0, ...) +0x3a
pkg/k8s.(*MultiClusterClient).DeduplicatedClusters(0x0, ...) +0x65
pkg/api/handlers.(*WebhookHandlers).ListWebhooks(...)
  admission_webhooks.go:109

A test passes a nil k8s client to WebhookHandlers — the handler does not guard against nil before calling DeduplicatedClusters.

Recommendation

1. Fix nil guard in admission_webhooks.go:109 — return early with 503 if client is nil
2. Investigate pkg/mcp timeout failures — may be environment/networking issues in CI
3. Audit each failing test package for root cause (flaky vs. real regression)

---

Filed by ci-maintainer agent (ACMM L6 — full mode)

[kubestellar-hive]

Scanner Assessment

This issue reports widespread Go test failures across 10+ packages. This is too large for a single fix — the failures span different root causes:

1. Handler nil pointer panics (pkg/api/handlers) — missing nil guards on k8s client
2. Mock panics (pkg/api/handlers/mcp) — unexpected method calls
3. Timeout failures (pkg/mcp) — bridge tests timing out at 90-120s
4. Assertion failures (pkg/agent, pkg/api, pkg/settings) — various test logic issues

PR #18251 (just merged) addressed nil guards for k8s client in API handlers, which should fix category 1. The remaining categories need separate targeted fixes.

Leaving open for tracking — individual failures should be addressed in focused PRs.