Skip to content

Namespace Management at Scale #9490

Description

@moukoublen

Problem

  • ECK only supports two namespace scoping modes: all namespaces, or a static hand-written list
  • The static list doesn't scale: adding/removing a tenant requires a config change + operator restart
  • No way to express policy like "manage namespaces labeled for Elastic" or "exclude system namespaces"
  • No exclusion primitive; users must enumerate the full positive set manually
  • Large multi-tenant customers (hundreds/thousands of namespaces with daily churn) are increasingly common

Proposed Solution

Introduce a namespaceSelector field (type metav1.LabelSelector) in the ECK operator configuration.

  • Replaces (but does not deprecate) the static namespaces list
  • Backward compatible: no selector = existing behavior unchanged
  • Mutually exclusive with namespaces; supplying both is a startup validation error
  • Evaluated against corev1.Namespace.Labels
  • Cluster-wide install profile required (createClusterScopedResources: true)

Implementation

  • Widen cache to cluster-wide; drop per-namespace scoping
  • Add a shared NamespaceMatcher service backed by a corev1.Namespace informer
  • Plug a NamespacePredicate into every controller's event filter
  • Add a Namespace controller that re-enqueues ECK CRs when a namespace's labels flip eligibility (Real-time dynamic evaluation)
  • Each Reconcile also gates on the namespace matcher at the top of the function
  • Cluster-scoped resources (webhooks, CRDs, etc.) short-circuit the predicate and always pass through
  • No operator restart needed; tenants on/offboarded purely via namespace relabeling

License Gating

Dynamic namespace selection is an Enterprise-tier feature (license.Checker.EnterpriseFeaturesEnabled).

Hybrid approach

  • Static snapshot (label-selector evaluation at startup) always works regardless of license
  • Real-time dynamic evaluation requires a valid enterprise license
  • Without license: compliance warning logged + emitted as K8s event; existing namespaces reconciled normally; new/deleted namespaces ignored until restart or license restored
  • On license renewal: real-time evaluation resumes + full re-sync triggered

Key Design Decisions

  • No cleanup on opt-out: namespace losing eligibility stops reconciliation, does not delete resources
  • One operator owns a namespace: disjoint selectors across operator instances are the admin's responsibility; no cross-operator arbitration
  • Labels managed externally: ECK reads namespace labels; it never writes or reconciles them
  • kubernetes.io/metadata.name available: name-based matching works via this GA label (since k8s 1.22)
  • Webhook behavior: silently allow requests from namespaces outside the operator's selector

Out of Scope

  • Lifecycle of Elastic resources within managed namespaces
  • RBAC for end users
  • Multi-operator coordination / conflict detection
  • Namespace-scoped install profile support for dynamic mode

Related Tickets:

Metadata

Metadata

Assignees

Labels

>enhancementEnhancement of existing functionality>featureAdds or discusses adding a feature to the product

Fields

No fields configured for Enhancement.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions