Support multiple operator instances in one k8s cluster

[mflendrich]

Make it possible to run multiple instances of Operator in the cluster.

There are two approaches that might work:

• for the namespaced resources, restrict the operator's controllers to a specific namespace
• for the non-namespaced resources, introduce the "class" concept akin to StorageClass, IngressClass, GatewayClass or a unique identifier.

Tasks

Preview Give feedback

1. https://github.com/scylladb/scylla-operator-enterprise/issues/58

Acceptance criteria

• It is possible to run multiple Operators in a way that each resource has a well-defined relationship to one operator it will be reconciled by.
• The multi-operator workflow is documented.

[scylla-operator-bot]

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 30d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out

/lifecycle stale