Skip to content

Commit 7ebcabc

Browse files
olavloiteolavloite
authored
feat: support Spanner PostgreSQL databases (#199)
* feat: support Spanner PostgreSQL databases Adds support for using the gorm PostgreSQL dialect with Spanner. Connections to Spanner use the Spanner database/sql driver, and not one of the PostgreSQL database drivers (e.g. it does not use pgx). Using this dialect with Spanner does not require the use of PGAdapter. * chore: address review comments * chore: update dependencies
1 parent 323dad1 commit 7ebcabc

61 files changed

Lines changed: 6503 additions & 4675 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/integration-tests-on-emulator.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,13 @@ jobs:
2424
run: go test -v
2525
env:
2626
SPANNER_EMULATOR_HOST: localhost:9010
27+
- name: Run PostgreSQL integration tests on Emulator
28+
run: go test -v
29+
working-directory: postgresql
30+
env:
31+
SPANNER_EMULATOR_HOST: localhost:9010
32+
- name: Run PostgreSQL sample application integration tests on Emulator
33+
run: go test -v
34+
working-directory: postgresql/samples/sample_application
35+
env:
36+
SPANNER_EMULATOR_HOST: localhost:9010

‎.github/workflows/integration-tests-on-production.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ jobs:
3333
with:
3434
credentials_json: ${{ secrets.GCP_SA_KEY }}
3535
- name: Run integration tests on production
36-
run: go test -v -timeout 45m ./... -run '^TestIntegration_'
36+
run: go test -v -timeout 45m -run '^TestIntegration_'
3737
env:
3838
JOB_TYPE: test
3939
SPANNER_TEST_PROJECT: ${{ secrets.GCP_PROJECT_ID }}
@@ -42,7 +42,7 @@ jobs:
4242
working-directory: ./samples/sample_application
4343
run: |
4444
go mod tidy
45-
go test -v -timeout 45m ./... -run '^TestIntegration_'
45+
go test -v -timeout 45m -run '^TestIntegration_'
4646
env:
4747
JOB_TYPE: test
4848
SPANNER_TEST_PROJECT: ${{ secrets.GCP_PROJECT_ID }}

‎.github/workflows/postgresql.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
on:
2+
push:
3+
branches: [ main ]
4+
pull_request:
5+
branches: [ main ]
6+
name: PostgreSQL-dialect tests
7+
jobs:
8+
emulator-tests:
9+
runs-on: ubuntu-latest
10+
services:
11+
emulator:
12+
image: gcr.io/cloud-spanner-emulator/emulator:latest
13+
ports:
14+
- 9010:9010
15+
- 9020:9020
16+
steps:
17+
- name: Install Go
18+
uses: actions/setup-go@v6
19+
with:
20+
go-version: 1.25.x
21+
- name: Checkout code
22+
uses: actions/checkout@v6
23+
- name: Checkout gorm
24+
uses: actions/checkout@v6
25+
with:
26+
repository: go-gorm/gorm
27+
path: postgresql/tests/gorm
28+
- name: Run gorm PostgreSQL tests on Emulator
29+
working-directory: postgresql/tests
30+
run: ./run_gorm_tests.sh
31+
env:
32+
SPANNER_EMULATOR_HOST: localhost:9010

‎.github/workflows/samples-postgresql.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
on:
2+
pull_request:
3+
branches: [ main ]
4+
name: Run Samples for PostgreSQL
5+
jobs:
6+
samples:
7+
strategy:
8+
matrix:
9+
go-version: [1.24.x, 1.25.x]
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Install Go
13+
uses: actions/setup-go@v6
14+
with:
15+
go-version: ${{ matrix.go-version }}
16+
- name: Checkout code
17+
uses: actions/checkout@v6
18+
- name: Run Hello World
19+
working-directory: ./postgresql/samples/helloworld
20+
run: go run main.go
21+
- name: Run Quickstart
22+
working-directory: ./postgresql/samples/quickstart
23+
run: go run main.go
24+
- name: Run sample application
25+
working-directory: ./postgresql/samples
26+
run: go run run_sample.go
27+
- name: Run snippets
28+
working-directory: ./postgresql/samples
29+
run: |
30+
for filename in snippets/*.go; do
31+
name=${filename##*/}
32+
base=${name%.go}
33+
echo "Running sample $base"
34+
go run run_sample.go "$base" || ! break
35+
done

‎README.md‎

Lines changed: 65 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,83 @@
1-
# go-gorm-spanner
1+
# gorm Dialect for Spanner
22

33
[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/googleapis/go-gorm-spanner)
44

5-
[Google Cloud Spanner](https://cloud.google.com/spanner) ORM for
6-
Go's [GORM](https://gorm.io/) implementation.
5+
[Google Cloud Spanner](https://cloud.google.com/spanner) ORM for Go's [GORM](https://gorm.io/) implementation.
6+
7+
This project contains `gorm` dialects for both Spanner GoogleSQL and Spanner PostgreSQL databases.
8+
9+
## GoogleSQL
10+
11+
This example shows how to connect `gorm` to a Spanner GoogleSQL database.
712

813
``` go
914
import (
10-
"gorm.io/gorm"
11-
_ "github.com/googleapis/go-sql-spanner"
15+
"fmt"
1216

13-
spannergorm "github.com/googleapis/go-gorm-spanner"
17+
spannergorm "github.com/googleapis/go-gorm-spanner"
18+
_ "github.com/googleapis/go-sql-spanner"
19+
"gorm.io/gorm"
1420
)
1521

16-
db, err := gorm.Open(spannergorm.New(spannergorm.Config{
17-
DriverName: "spanner",
18-
DSN: "projects/PROJECT/instances/INSTANCE/databases/DATABASE",
19-
}), &gorm.Config{PrepareStmt: true})
20-
if err != nil {
21-
log.Fatal(err)
22+
func helloWorld(projectId, instanceId, databaseId string) error {
23+
db, err := gorm.Open(spannergorm.New(spannergorm.Config{
24+
DriverName: "spanner",
25+
DSN: fmt.Sprintf("projects/%s/instances/%s/databases/%s", projectId, instanceId, databaseId),
26+
}), &gorm.Config{PrepareStmt: true})
27+
if err != nil {
28+
return fmt.Errorf("failed to open database connection: %v\n", err)
29+
}
30+
var msg string
31+
if err := db.Raw("SELECT 'Hello World!'").Scan(&msg).Error; err != nil {
32+
return fmt.Errorf("failed to execute query: %v", err)
33+
}
34+
fmt.Println(msg)
35+
36+
return nil
2237
}
38+
```
2339

24-
// Print singers with more than 500 likes.
25-
type Singer struct {
26-
gorm.Model
27-
Text string
28-
Likes int
29-
}
30-
var singers []Singer
31-
if err := db.Where("likes > ?", 500).Find(&singers).Error; err != nil {
32-
log.Fatal(err)
33-
}
34-
for s := range singers {
35-
fmt.Println(s.ID, s.Text)
40+
See the [samples directory](samples) for more examples.
41+
42+
## PostgreSQL
43+
44+
This example shows how to connect `gorm` to a Spanner PostgreSQL database.
45+
46+
``` go
47+
import (
48+
"fmt"
49+
50+
spannerpg "github.com/googleapis/go-gorm-spanner/postgresql"
51+
_ "github.com/googleapis/go-sql-spanner"
52+
"gorm.io/driver/postgres"
53+
"gorm.io/gorm"
54+
)
55+
56+
func helloWorld(projectId, instanceId, databaseId string) error {
57+
db, err := gorm.Open(spannerpg.New(postgres.Config{
58+
DSN: fmt.Sprintf("projects/%s/instances/%s/databases/%s", projectId, instanceId, databaseId),
59+
}), &gorm.Config{})
60+
if err != nil {
61+
return fmt.Errorf("failed to open database connection: %v\n", err)
62+
}
63+
var msg string
64+
if err := db.Raw("SELECT $1::varchar as greeting", "Hello World from Spanner PostgreSQL!").Scan(&msg).Error; err != nil {
65+
return fmt.Errorf("failed to execute query: %v", err)
66+
}
67+
fmt.Println(msg)
68+
69+
return nil
3670
}
3771
```
3872

39-
### Connection URL Properties
73+
See the [PostgreSQL samples directory](postgresql/samples) for more examples for how to use this library with a Spanner
74+
PostgreSQL database.
75+
76+
## Connection URL Properties
4077

4178
The Cloud Spanner GORM supports the following connection URL properties
4279

43-
#### Commonly Used Properties
80+
### Commonly Used Properties
4481
- credentials (String): File name for the credentials to use. The connection will use the default credentials of the environment if no credentials file is specified in the connection string. Example: `projects/my-project/instances/my-instance/databases/my-db;credentials=/path/to/credentials.json`
4582
- optimizerVersion (String): Sets the default query optimizer version to use for this connection. See also https://cloud.google.com/spanner/docs/query-optimizer/query-optimizer-versions.
4683
- isolationLevel (String): Sets the default isolation level for read/write transaction. The default is `sql.LevelSerializable`. Other supported values are `sql.LevelRepeatableRead`. Example: `fmt.Sprintf("projects/my-project/instances/my-instance/databases/my-db;isolationLevel=%s", sql.LevelRepeatableRead)`
@@ -51,15 +88,15 @@ minimum and maximum number of sessions for `gorm`.
5188

5289
See https://cloud.google.com/spanner/docs/sessions#multiplexed_sessions for more information about multiplexed sessions.
5390

54-
#### Advanced Properties
91+
### Advanced Properties
5592
- numChannels (int): Sets the number of gRPC channels to use. Defaults to 4.
5693
- retryAbortsInternally (boolean): Boolean that indicates whether the connection should automatically retry aborted errors. The default is true.
5794
- disableRouteToLeader (boolean): Boolean that indicates if all the requests of type read-write and PDML need to be routed to the leader region. The default is false.
5895
- usePlainText (boolean): : Boolean that indicates whether the connection should use plain text communication or not. Set this to true to connect to local mock servers that do not use SSL. Example: `projects/test-project/instances/test-instance/databases/test-db;usePlainText=true`
5996

6097
Example: `projects/my-project/instances/my-instance/databases/my-db;numChannels=4;retryAbortsInternally=true;disableRouteToLeader=false;usePlainText=false`
6198

62-
#### Additional Spanner Configuration
99+
### Additional Spanner Configuration
63100
You can also connect `gorm` to Spanner using a `driver.Connector`. This allows you to supply additional configuration
64101
for the Spanner client that should be used for `gorm`:
65102

‎array_types_test.go‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,8 @@ func TestInsertNativeArrays(t *testing.T) {
7676
if g, w := len(req.Params.Fields), 9; g != w {
7777
t.Errorf("num params mismatch\n Got: %v\nWant: %v", g, w)
7878
}
79-
if g, w := len(req.ParamTypes), 9; g != w {
79+
// String arrays are sent as untyped values to allow Spanner to infer the data type from the SQL statement.
80+
if g, w := len(req.ParamTypes), 8; g != w {
8081
t.Errorf("num param types mismatch\n Got: %v\nWant: %v", g, w)
8182
}
8283
wantParamTypes := []spannerpb.TypeCode{
@@ -86,13 +87,19 @@ func TestInsertNativeArrays(t *testing.T) {
8687
spannerpb.TypeCode_INT64,
8788
spannerpb.TypeCode_FLOAT32,
8889
spannerpb.TypeCode_FLOAT64,
89-
spannerpb.TypeCode_STRING,
90+
spannerpb.TypeCode_TYPE_CODE_UNSPECIFIED,
9091
spannerpb.TypeCode_TIMESTAMP,
9192
}
9293
for i, code := range wantParamTypes {
9394
param := fmt.Sprintf("p%d", i+1)
94-
if g, w := req.ParamTypes[param].ArrayElementType.Code, code; g != w {
95-
t.Errorf("%s: param type mismatch\n Got: %v\nWant: %v", param, g, w)
95+
if tp, ok := req.ParamTypes[param]; ok {
96+
if g, w := tp.ArrayElementType.Code, code; g != w {
97+
t.Errorf("%s: param type mismatch\n Got: %v\nWant: %v", param, g, w)
98+
}
99+
} else {
100+
if g, w := spannerpb.TypeCode_TYPE_CODE_UNSPECIFIED, code; g != w {
101+
t.Errorf("%s: param type mismatch\n Got: %v\nWant: %v", param, g, w)
102+
}
96103
}
97104
}
98105
wantValues := []string{

‎benchmarks/go.mod‎

Lines changed: 32 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -7,66 +7,65 @@ toolchain go1.25.5
77
replace github.com/googleapis/go-gorm-spanner => ../
88

99
require (
10-
cloud.google.com/go/spanner v1.87.0
10+
cloud.google.com/go/spanner v1.88.0
1111
github.com/google/uuid v1.6.0
12-
github.com/googleapis/go-gorm-spanner v1.9.0
13-
github.com/googleapis/go-sql-spanner v1.22.0
14-
google.golang.org/genproto v0.0.0-20251222181119-0a764e51fe1b
12+
github.com/googleapis/go-gorm-spanner v1.9.1
13+
github.com/googleapis/go-sql-spanner v1.23.0
14+
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409
1515
google.golang.org/grpc v1.78.0
1616
gorm.io/datatypes v1.2.7
1717
gorm.io/gorm v1.31.1
1818
)
1919

2020
require (
21-
cel.dev/expr v0.24.0 // indirect
21+
cel.dev/expr v0.25.1 // indirect
2222
cloud.google.com/go v0.123.0 // indirect
23-
cloud.google.com/go/auth v0.17.0 // indirect
23+
cloud.google.com/go/auth v0.18.1 // indirect
2424
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
2525
cloud.google.com/go/compute/metadata v0.9.0 // indirect
2626
cloud.google.com/go/iam v1.5.3 // indirect
27-
cloud.google.com/go/longrunning v0.7.0 // indirect
27+
cloud.google.com/go/longrunning v0.8.0 // indirect
2828
cloud.google.com/go/monitoring v1.24.3 // indirect
2929
filippo.io/edwards25519 v1.1.0 // indirect
30-
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.3 // indirect
30+
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.6.0 // indirect
3131
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 // indirect
3232
github.com/cespare/xxhash/v2 v2.3.0 // indirect
33-
github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f // indirect
34-
github.com/envoyproxy/go-control-plane/envoy v1.35.0 // indirect
35-
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
33+
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 // indirect
34+
github.com/envoyproxy/go-control-plane/envoy v1.36.0 // indirect
35+
github.com/envoyproxy/protoc-gen-validate v1.3.0 // indirect
3636
github.com/felixge/httpsnoop v1.0.4 // indirect
3737
github.com/go-jose/go-jose/v4 v4.1.3 // indirect
3838
github.com/go-logr/logr v1.4.3 // indirect
3939
github.com/go-logr/stdr v1.2.2 // indirect
40-
github.com/go-sql-driver/mysql v1.8.1 // indirect
41-
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
40+
github.com/go-sql-driver/mysql v1.9.3 // indirect
4241
github.com/google/s2a-go v0.1.9 // indirect
43-
github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect
44-
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
42+
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
43+
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
4544
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
4645
github.com/jinzhu/inflection v1.0.0 // indirect
4746
github.com/jinzhu/now v1.1.5 // indirect
4847
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
4948
github.com/spiffe/go-spiffe/v2 v2.6.0 // indirect
5049
go.opencensus.io v0.24.0 // indirect
5150
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
52-
go.opentelemetry.io/contrib/detectors/gcp v1.38.0 // indirect
53-
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.62.0 // indirect
54-
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 // indirect
55-
go.opentelemetry.io/otel v1.38.0 // indirect
56-
go.opentelemetry.io/otel/metric v1.38.0 // indirect
57-
go.opentelemetry.io/otel/sdk v1.38.0 // indirect
58-
go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect
59-
go.opentelemetry.io/otel/trace v1.38.0 // indirect
60-
golang.org/x/crypto v0.45.0 // indirect
61-
golang.org/x/net v0.47.0 // indirect
62-
golang.org/x/oauth2 v0.33.0 // indirect
63-
golang.org/x/sync v0.18.0 // indirect
64-
golang.org/x/sys v0.38.0 // indirect
65-
golang.org/x/text v0.31.0 // indirect
51+
go.opentelemetry.io/contrib/detectors/gcp v1.39.0 // indirect
52+
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.64.0 // indirect
53+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 // indirect
54+
go.opentelemetry.io/otel v1.39.0 // indirect
55+
go.opentelemetry.io/otel/metric v1.39.0 // indirect
56+
go.opentelemetry.io/otel/sdk v1.39.0 // indirect
57+
go.opentelemetry.io/otel/sdk/metric v1.39.0 // indirect
58+
go.opentelemetry.io/otel/trace v1.39.0 // indirect
59+
golang.org/x/crypto v0.47.0 // indirect
60+
golang.org/x/net v0.49.0 // indirect
61+
golang.org/x/oauth2 v0.35.0 // indirect
62+
golang.org/x/sync v0.19.0 // indirect
63+
golang.org/x/sys v0.40.0 // indirect
64+
golang.org/x/text v0.33.0 // indirect
6665
golang.org/x/time v0.14.0 // indirect
67-
google.golang.org/api v0.257.0 // indirect
68-
google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2 // indirect
69-
google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 // indirect
66+
google.golang.org/api v0.266.0 // indirect
67+
google.golang.org/genproto/googleapis/api v0.0.0-20260203192932-546029d2fa20 // indirect
68+
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
7069
google.golang.org/protobuf v1.36.11 // indirect
71-
gorm.io/driver/mysql v1.5.6 // indirect
70+
gorm.io/driver/mysql v1.6.0 // indirect
7271
)

0 commit comments

Comments
 (0)