feat: add GCP Cloud Spanner scaler#7844
Open
n0rm4l-me wants to merge 5 commits into
Open
Conversation
Introduces a new `gcp-spanner` trigger that scales workloads based on the result of an arbitrary SQL query executed against a Cloud Spanner database. The query must return a single INT64 value in the first column (typically COUNT(*) or SUM(...)). Key design decisions: - Empty result sets (iterator.Done) are treated as value=0, not errors, so both aggregate queries and single-value SELECTs work correctly. - When SPANNER_EMULATOR_HOST is set, credential validation is skipped so the scaler works out-of-the-box against a local emulator. - Supports all three GCP auth methods: inline JSON, env-file path, and Workload Identity via pod identity provider. Files added: - pkg/scalers/gcp_spanner_scaler.go — scaler implementation - pkg/scalers/gcp_spanner_scaler_test.go — 14 unit tests - pkg/scalers/gcp_spanner_scaler_integration_test.go — 6 integration tests - tests/scalers/gcp/gcp_spanner/gcp_spanner_test.go — e2e test - scripts/run-spanner-emulator.sh — helper for local dev Files modified: - pkg/scaling/scalers_builder.go — register "gcp-spanner" case - go.mod / go.sum / vendor/ — add cloud.google.com/go/spanner - CHANGELOG.md — Unreleased entry Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
- Replace deprecated option.WithCredentialsJSON/WithCredentialsFile with option.WithAuthCredentialsJSON/WithAuthCredentialsFile (staticcheck SA1019) - Rename unused ctx parameter to _ in Close() (revive unused-parameter) - Remove unnecessary tc := tc loop-variable copies (copyloopvar, Go 1.22+) - Remove no-op fmt.Sprintf in e2e test var block (staticcheck S1039) - Fix gci import/var alignment in e2e test - Run gofmt on integration test file Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
|
Thank you for your contribution! 🙏 Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected. While you are waiting, make sure to:
Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient. Learn more about our contribution guide. |
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
gcp-spanner must come before gcp-stackdriver (sp < st). Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
Author
|
@wozniakjan Could you please review? |
|
Nice, I would like to have it! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new
gcp-spannertrigger that scales Kubernetes workloads based on the result of an arbitrary SQL query against a Cloud Spanner database.The query must return a single row whose first column is an
INT64value — typicallyCOUNT(*)orSUM(...)representing current workload depth (e.g. number of pending jobs).Motivation
Cloud Spanner is a globally-distributed relational database widely used for high-throughput transactional workloads. A common pattern is a job-queue table in Spanner where workers poll for work. This scaler enables autoscaling those workers directly from queue depth without an intermediary metrics pipeline.
Changes
New files:
pkg/scalers/gcp_spanner_scaler.go— scaler implementationpkg/scalers/gcp_spanner_scaler_test.go— 14 unit tests (all required fields, all auth methods, defaults, metric name generation)pkg/scalers/gcp_spanner_scaler_integration_test.go— 6 integration tests against the Cloud Spanner emulator (run with-tags integration)tests/scalers/gcp/gcp_spanner/gcp_spanner_test.go— e2e test (activation threshold, scale-out, scale-in)scripts/run-spanner-emulator.sh— helper script to start the emulator locallyModified files:
pkg/scaling/scalers_builder.go— registergcp-spannercase (alphabetical order maintained)go.mod/go.sum/vendor/— addcloud.google.com/go/spannerCHANGELOG.md— entry underUnreleased → NewScaler parameters
projectIdinstanceIddatabaseIdqueryINT64valuetargetValue5activationValue0credentialsFromEnvcredentialsFromEnvFileGoogleApplicationCredentials(authParams)GCP Workload Identity (pod identity provider
gcp) is also supported.Design notes
SELECTwith no matching records), the scaler treats the value as0rather than returning an error. This makes arbitrary single-valueSELECTqueries safe in addition toCOUNT(*)aggregates.SPANNER_EMULATOR_HOSTis set, credential validation is skipped — the Spanner SDK handles insecure auth automatically.option.WithAuthCredentialsJSON/option.WithAuthCredentialsFile(non-deprecated API), consistent withgcp_storage_scaler.go.Testing
Unit tests (no external dependencies):
Integration tests (requires Cloud Spanner emulator):
e2e tests: require
TF_GCP_SA_CREDENTIALS,TF_GCP_SPANNER_INSTANCE_ID,TF_GCP_SPANNER_DATABASE_IDenv vars and a running Kubernetes cluster with KEDA deployed.Documentation
Docs PR: kedacore/keda-docs#1786
Checklist
go fmt/golangci-lintclean