Skip to content

feat: add GCP Cloud Spanner scaler#7844

Open
n0rm4l-me wants to merge 5 commits into
kedacore:mainfrom
n0rm4l-me:feat/gcp-spanner-scaler
Open

feat: add GCP Cloud Spanner scaler#7844
n0rm4l-me wants to merge 5 commits into
kedacore:mainfrom
n0rm4l-me:feat/gcp-spanner-scaler

Conversation

@n0rm4l-me

@n0rm4l-me n0rm4l-me commented Jun 4, 2026

Copy link
Copy Markdown

Description

Adds a new gcp-spanner trigger that scales Kubernetes workloads based on the result of an arbitrary SQL query against a Cloud Spanner database.

The query must return a single row whose first column is an INT64 value — typically COUNT(*) or SUM(...) representing current workload depth (e.g. number of pending jobs).

Motivation

Cloud Spanner is a globally-distributed relational database widely used for high-throughput transactional workloads. A common pattern is a job-queue table in Spanner where workers poll for work. This scaler enables autoscaling those workers directly from queue depth without an intermediary metrics pipeline.

Changes

New files:

  • pkg/scalers/gcp_spanner_scaler.go — scaler implementation
  • pkg/scalers/gcp_spanner_scaler_test.go — 14 unit tests (all required fields, all auth methods, defaults, metric name generation)
  • pkg/scalers/gcp_spanner_scaler_integration_test.go — 6 integration tests against the Cloud Spanner emulator (run with -tags integration)
  • tests/scalers/gcp/gcp_spanner/gcp_spanner_test.go — e2e test (activation threshold, scale-out, scale-in)
  • scripts/run-spanner-emulator.sh — helper script to start the emulator locally

Modified files:

  • pkg/scaling/scalers_builder.go — register gcp-spanner case (alphabetical order maintained)
  • go.mod / go.sum / vendor/ — add cloud.google.com/go/spanner
  • CHANGELOG.md — entry under Unreleased → New

Scaler parameters

Parameter Required Default Description
projectId GCP project ID
instanceId Spanner instance ID
databaseId Spanner database ID
query SQL returning a single INT64 value
targetValue 5 Metric value per replica
activationValue 0 Minimum value to activate the scaler
credentialsFromEnv one of three Env var name containing service account JSON
credentialsFromEnvFile one of three Env var name containing path to JSON key file
GoogleApplicationCredentials (authParams) one of three Inline service account JSON via TriggerAuthentication

GCP Workload Identity (pod identity provider gcp) is also supported.

Design notes

  • Empty result set → value 0: if the query returns no rows (e.g. a SELECT with no matching records), the scaler treats the value as 0 rather than returning an error. This makes arbitrary single-value SELECT queries safe in addition to COUNT(*) aggregates.
  • Emulator support: when SPANNER_EMULATOR_HOST is set, credential validation is skipped — the Spanner SDK handles insecure auth automatically.
  • Uses option.WithAuthCredentialsJSON / option.WithAuthCredentialsFile (non-deprecated API), consistent with gcp_storage_scaler.go.

Testing

Unit tests (no external dependencies):

go test -v -run TestSpanner ./pkg/scalers/

Integration tests (requires Cloud Spanner emulator):

# Start emulator
./scripts/run-spanner-emulator.sh

# Run tests
SPANNER_EMULATOR_HOST=localhost:9010 go test -tags integration -v -run TestSpannerIntegration ./pkg/scalers/

e2e tests: require TF_GCP_SA_CREDENTIALS, TF_GCP_SPANNER_INSTANCE_ID, TF_GCP_SPANNER_DATABASE_ID env vars and a running Kubernetes cluster with KEDA deployed.

Documentation

Docs PR: kedacore/keda-docs#1786

Checklist

n0rm4l-me added 2 commits June 4, 2026 11:57
Introduces a new `gcp-spanner` trigger that scales workloads based on
the result of an arbitrary SQL query executed against a Cloud Spanner
database.  The query must return a single INT64 value in the first
column (typically COUNT(*) or SUM(...)).

Key design decisions:
- Empty result sets (iterator.Done) are treated as value=0, not errors,
  so both aggregate queries and single-value SELECTs work correctly.
- When SPANNER_EMULATOR_HOST is set, credential validation is skipped
  so the scaler works out-of-the-box against a local emulator.
- Supports all three GCP auth methods: inline JSON, env-file path, and
  Workload Identity via pod identity provider.

Files added:
- pkg/scalers/gcp_spanner_scaler.go          — scaler implementation
- pkg/scalers/gcp_spanner_scaler_test.go     — 14 unit tests
- pkg/scalers/gcp_spanner_scaler_integration_test.go — 6 integration tests
- tests/scalers/gcp/gcp_spanner/gcp_spanner_test.go  — e2e test
- scripts/run-spanner-emulator.sh            — helper for local dev

Files modified:
- pkg/scaling/scalers_builder.go             — register "gcp-spanner" case
- go.mod / go.sum / vendor/                  — add cloud.google.com/go/spanner
- CHANGELOG.md                               — Unreleased entry

Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
- Replace deprecated option.WithCredentialsJSON/WithCredentialsFile with
  option.WithAuthCredentialsJSON/WithAuthCredentialsFile (staticcheck SA1019)
- Rename unused ctx parameter to _ in Close() (revive unused-parameter)
- Remove unnecessary tc := tc loop-variable copies (copyloopvar, Go 1.22+)
- Remove no-op fmt.Sprintf in e2e test var block (staticcheck S1039)
- Fix gci import/var alignment in e2e test
- Run gofmt on integration test file

Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
@n0rm4l-me n0rm4l-me requested a review from a team as a code owner June 4, 2026 03:22
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

Thank you for your contribution! 🙏

Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected.

While you are waiting, make sure to:

  • Add an entry in our changelog in alphabetical order and link related issue
  • Update the documentation, if needed
  • Add unit & e2e tests for your changes
  • GitHub checks are passing
  • Is the DCO check failing? Here is how you can fix DCO issues

Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient.

Learn more about our contribution guide.

@keda-automation keda-automation requested a review from a team June 4, 2026 03:23
@snyk-io

snyk-io Bot commented Jun 4, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
n0rm4l-me added 2 commits June 4, 2026 12:29
Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
gcp-spanner must come before gcp-stackdriver (sp < st).

Signed-off-by: Petr Petrenko | INPD <petr.petrenko@rakuten.com>
@n0rm4l-me

Copy link
Copy Markdown
Author

@wozniakjan Could you please review?

@Reese-Reese-123

Copy link
Copy Markdown

Nice, I would like to have it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants