What would you like to be added:

Add ability to add metadata based on Clusterqueue labels/annotations for prometheus metrics like: kueue_cluster_queue_resource_usage. These can be added for any kueue_cluster_queue_* or even extended to other metrics if useful.

Why is this needed:

It is useful to get custom metadata into the Clusterqueue metrics as labels which could make filtering easier for a variety of use cases.

As an example, let us say I want to calculate sum of usage for clusterqueues aggregated over some custom label, example all clusterqueues by parentCohort, or clusterqueues belonging to one team/organization that is available as a label in Clusterqueue metadata.

Completion requirements:
kueue_cluster_queue_* metrics including kueue_cluster_queue_resource_usage have a label which could have values injected from clusterqueue metadata.

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.