⭐️ staged k8s discovery

[imilchev]

Summary

• Introduces toggle-based staged K8s discovery that splits the single-pass discovery into two phases: cluster+namespaces first, workloads per namespace later
• Maintains full backward compatibility — old clients without the toggle get the original single-pass behavior unchanged
• Defines OptionStagedDiscovery in providers-sdk/v1/plugin so any provider can opt in to staged discovery
• mql CLI automatically sets the staged discovery option on all provider connections

Motivation

K8s discovery currently runs everything in a single pass: cluster + all namespaces + all workloads in every namespace. For large clusters this is expensive and doesn't allow clients to control the discovery granularity.

Staged discovery lets clients discover cluster and namespaces first, then connect to individual namespaces to discover their workloads on demand. This enables:

• Reduced initial discovery cost for large clusters
• Per-namespace parallelism on the client side
• Independent resource caches per namespace (no stale data from shared caches)

Design

Toggle mechanism: The plugin.OptionStagedDiscovery key is set on the connection config's Options map. Providers check for its presence (not value) to decide which path to take.

Stage routing: When staged discovery is enabled, the provider checks whether OPTION_NAMESPACE is already set on the config:

• No namespace → Stage 1 (discoverClusterStage): returns cluster asset + namespace assets WITH platform IDs and discovery targets. Each namespace's connection config has OPTION_NAMESPACE set.
• Namespace present → Stage 2 (discoverNamespaceStage): returns only workloads for that namespace. The namespace asset was already emitted by stage 1.

Legacy path: When OptionStagedDiscovery is absent (old clients), discoverLegacy runs the original single-pass logic unchanged.

Changes

• providers-sdk/v1/plugin/connection.go — Added OptionStagedDiscovery constant, reusable across all providers
• providers/k8s/resources/discovery.go — Refactored Discover() into three paths: discoverLegacy(), discoverClusterStage(), discoverNamespaceStage()
• cli/providers/providers.go — Sets OptionStagedDiscovery on all connection configs from CLI commands

Test plan

• Legacy path: mql run k8s -c "k8s.deployments { name namespace }" works identically to main
• Legacy path: mql run k8s --discover namespaces -c "asset.name" returns namespaces
• Namespace filter: mql run k8s --namespaces kube-system -c "k8s.pods { name }" works in both paths
• go test ./providers/k8s/... passes

🤖 Generated with Claude Code

[mondoo-code-review]

K8s discovery is split into two stages (cluster → namespace) to give each namespace its own resource cache, fixing stale data when scanning multiple namespaces.

[github-actions]

Test Results

5 466 tests  ±0   5 462 ✅ ±0   2m 32s ⏱️ +30s
  411 suites ±0       4 💤 ±0 
   31 files   ±0       0 ❌ ±0 

Results for commit 8714b25. ± Comparison against base commit a8d4086.

♻️ This comment has been updated with latest results.

[mondoo-code-review]

Staged K8s discovery refactored to use a global opt-in toggle with backward-compatible legacy path.

[imilchev]

this is now included in #6949