Mimir multi-tenancy

[NRCan-LGariepy]

Hi everyone, I have a conceptual question regarding Mimir and multi-tenant metrics.

Context: We have many different teams using our kubernetes cluster as a platform. For now the teams are mostly internal, but it's a real possibility that at some point we have external teams using our cluster as well. We've been investigating putting in place metrics multi-tenancy to prepare for that. Currently all metrics are scraped by prometheus-operator and sent to the default mimir tenant.

Question: We'd like to provide tenants with metrics regarding the namespaces they own, for example, how many pods they have, what pods are running, how much CPU they are using compared to the namespace quota, etc. I'm a bit unsure how to implement that though. If I split those metrics into tenants, do I lose the ability to have a global overview of all metrics? Will all my global dashboards break? WIll I need to duplicate data between the default (global) tenant and all the team tenants?

Basically I'd appreciate some guidance from the people who went through the process of implementing multi-tenancy. Thanks!

[deniszh]

If I split those metrics into tenants, do I lose the ability to have a global overview of all metrics?

Yes, you do. You can use something like promxy to merge data but it's separate component and need to be configured and maintained.

Will all my global dashboards break?

Yes, they will.

WIll I need to duplicate data between the default (global) tenant and all the team tenants?

Yes, that's the way. Loki has migration tool to split tenants but Mimir do not have it, so, there's no easy way to migrate historical data.