ScyllaClusters connected into a multi-datacenter cluster with `externalSeeds` are registered as separate clusters with Scylla Manager

[rzetelskik]

@Michal-Leszczynski discovered that when we useexternalSeeds to connect two ScyllaClusters (datacenters) into a single, multi-dc cluster, both ScyllaClusters are being registered with Scylla Manager as separate clusters with distinct IDs by the manager controller. It can be observed e.g. in the e2e test for external seeds: https://github.com/scylladb/scylla-operator/blob/master/test/e2e/set/scyllacluster/scyllacluster_external_seeds.go.

It's an issue since it can possibly break some of the operations performed by manager, e.g. the two registered clusters wouldn't be protected from running a backup simultaneously.

We should only register all ScyllaClusters, being part of the same logical cluster, with manager once, and the same ID should be propagated to their statuses.

Since the DCs share a common cluster name, this can most likely be fixed as part of #1902, but we should consider adding a dedicated e2e test verifying that the clusters are indeed only registered once.

/priority important-longterm
/kind bug
/assign

[scylla-operator-bot]

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 30d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out

/lifecycle stale

[tnozicka]

/remove-lifecycle stale
/triage accepted