Skip to content

Add flag to enable high-cardinality HTTP duration labels#7751

Open
aliaqel-stripe wants to merge 3 commits into
kedacore:mainfrom
aliaqel-stripe:feat/7731-enable-high-cardinality-metrics
Open

Add flag to enable high-cardinality HTTP duration labels#7751
aliaqel-stripe wants to merge 3 commits into
kedacore:mainfrom
aliaqel-stripe:feat/7731-enable-high-cardinality-metrics

Conversation

@aliaqel-stripe

@aliaqel-stripe aliaqel-stripe commented May 20, 2026

Copy link
Copy Markdown
Contributor

Provide a description of what has been changed

This adds --enable-high-cardinality-metrics-labels to the KEDA operator, defaulting to false.

When the flag is enabled, keda_scaler_http_request_duration_seconds includes the high-cardinality namespace, scaled_resource, trigger_name, and metric_name labels in addition to the existing low-cardinality labels. When the flag is disabled, the histogram is still emitted with only scaler and status_code labels.

The existing scaler HTTP request counter labels are unchanged, and the internal Metrics Service gRPC histograms are no longer gated by this flag.

This also updates unit and e2e coverage for both label modes and updates the changelog.

The e2e coverage was also hardened so the new flag assertions are reliable:

  • deployment arg mutations now wait for a full rollout, so the tests do not assert against old operator pods after toggling the flag
  • the OpenTelemetry disabled-path assertion now runs before the Prometheus HTTP-client scenarios, so the collector validates the low-cardinality histogram shape before the high-cardinality path runs
  • the OpenTelemetry scaler error assertion now waits for the collector export instead of relying on a fixed sleep

Checklist

  • I have verified that my change is according to the deprecations & breaking changes policy
  • Tests have been added (if applicable)
  • Changelog has been updated and is aligned with our changelog requirements, only when the change impacts end users
  • A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
  • A PR is opened to update the documentation on (repo) (if applicable)
  • Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #7731

Relates to kedacore/keda-docs#1772, kedacore/charts#855

@aliaqel-stripe aliaqel-stripe requested a review from a team as a code owner May 20, 2026 20:15
@github-actions

Copy link
Copy Markdown

Thank you for your contribution! 🙏

Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected.

While you are waiting, make sure to:

  • Add an entry in our changelog in alphabetical order and link related issue
  • Update the documentation, if needed
  • Add unit & e2e tests for your changes
  • GitHub checks are passing
  • Is the DCO check failing? Here is how you can fix DCO issues

Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient.

Learn more about our contribution guide.

@keda-automation keda-automation requested a review from a team May 20, 2026 20:15
@snyk-io

snyk-io Bot commented May 20, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Comment thread tests/helper/helper.go Outdated

require.Truef(t, updated, "container %s not found in deployment %s/%s", containerName, namespace, name)

_, err = kc.AppsV1().Deployments(namespace).Update(context.TODO(), deployment, metav1.UpdateOptions{})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep identified an issue in your code:
Consider to use well-defined context

To resolve this comment:

🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.

💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by context-todo.

You can view more details about this finding in the Semgrep AppSec Platform.

Comment thread tests/helper/helper.go
func SetDeploymentContainerArg(t *testing.T, kc *kubernetes.Clientset, name, namespace, containerName, flagName, flagValue string) {
t.Helper()

deployment, err := kc.AppsV1().Deployments(namespace).Get(context.TODO(), name, metav1.GetOptions{})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep identified an issue in your code:
Consider to use well-defined context

To resolve this comment:

🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.

💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by context-todo.

You can view more details about this finding in the Semgrep AppSec Platform.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ar test code

@aliaqel-stripe aliaqel-stripe changed the title Add flag to disable high-cardinality metrics Add flag to enable high-cardinality HTTP duration labels May 27, 2026
Signed-off-by: Ali Aqel <aliaqel@stripe.com>
Signed-off-by: Ali Aqel <aliaqel@stripe.com>
Signed-off-by: Ali Aqel <aliaqel@stripe.com>
@aliaqel-stripe aliaqel-stripe force-pushed the feat/7731-enable-high-cardinality-metrics branch from b9cd055 to 5e00a93 Compare May 27, 2026 20:05
@keda-automation keda-automation requested a review from a team May 27, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add flag to enable high cardinality metrics labels

2 participants