Skip to content

feat(etcd): use testcontainers-go for etcd tests #1721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
+307 −83
Draft

feat(etcd): use testcontainers-go for etcd tests #1721

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ca1046c
feat(etcd): use testcontainers-go for etcd tests
mdelapenya Apr 29, 2025
906daff
Merge branch 'main' into etcd-tc-go
gaby Apr 29, 2025
20f0a3b
Merge branch 'main' into etcd-tc-go
mdelapenya Apr 30, 2025
3f42f63
Merge branch 'main' into etcd-tc-go
gaby May 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,18 @@ jobs:
docker run -d -p 1408:1408 -p 30000:30000 ghcr.io/oracle/coherence-ce:22.06.5
sleep 30

- name: Install etcd
if: ${{ matrix.package == 'etcd' }}
- name: Setup Redis
if: ${{ matrix.package == 'rueidis' || matrix.package == 'valkey' }}
uses: shogo82148/actions-setup-redis@v1
with:
redis-version: '7.x'
auto-start: 'false'

- name: Run Redis
if: ${{ matrix.package == 'rueidis' || matrix.package == 'valkey' }}
run: |
docker run -d --name Etcd-server \
--publish 2379:2379 \
--publish 2380:2380 \
--env ALLOW_NONE_AUTHENTICATION=yes \
--env ETCD_ADVERTISE_CLIENT_URLS=http://etcd-server:2379 \
bitnami/etcd:latest
redis-server --port 6379 &
sleep 15

- name: Run Benchmarks
working-directory: ${{ matrix.package }}
Expand All @@ -112,6 +115,7 @@ jobs:
TEST_CLICKHOUSE_IMAGE: "clickhouse/clickhouse-server:23-alpine"
TEST_COUCHBASE_IMAGE: "couchbase:enterprise-7.6.5"
TEST_DYNAMODB_IMAGE: amazon/dynamodb-local:latest
TEST_ETCD_IMAGE: gcr.io/etcd-development/etcd:v3.5.21
TEST_MINIO_IMAGE: "docker.io/minio/minio:latest"
TEST_MONGODB_IMAGE: "docker.io/mongo:7"
TEST_MYSQL_IMAGE: "docker.io/mysql:9"
Expand Down
15 changes: 4 additions & 11 deletions .github/workflows/test-etcd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,24 +15,17 @@ jobs:
strategy:
matrix:
go-version:
- 1.19.x
- 1.20.x
- 1.21.x
- 1.23.x
- 1.24.x
steps:
- name: Fetch Repository
uses: actions/checkout@v4
- name: Install etcd
run: |
docker run -d --name Etcd-server \
--publish 2379:2379 \
--publish 2380:2380 \
--env ALLOW_NONE_AUTHENTICATION=yes \
--env ETCD_ADVERTISE_CLIENT_URLS=http://etcd-server:2379 \
bitnami/etcd:latest

- name: Install Go
uses: actions/setup-go@v5
with:
go-version: '${{ matrix.go-version }}'
- name: Run Test
env:
TEST_ETCD_IMAGE: gcr.io/etcd-development/etcd:v3.5.21
run: cd ./etcd && go test ./... -v -race
2 changes: 1 addition & 1 deletion etcd/etcd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"context"
"time"

"go.etcd.io/etcd/client/v3"
clientv3 "go.etcd.io/etcd/client/v3"
)

type Storage struct {
Expand Down
68 changes: 59 additions & 9 deletions etcd/etcd_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,25 +1,44 @@
package etcd

import (
"context"
"os"
"testing"
"time"

"github.com/stretchr/testify/require"

"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/modules/etcd"
)

var testStore *Storage
const (
// etcdImage is the default image used for running etcd in tests.
etcdImage = "gcr.io/etcd-development/etcd:v3.5.21"
etcdImageEnvVar string = "TEST_ETCD_IMAGE"
)

func TestMain(m *testing.M) {
testStore = New(Config{
Endpoints: []string{"localhost:2379"},
})
func newTestStore(t testing.TB) *Storage {
t.Helper()

img := etcdImage
if imgFromEnv := os.Getenv(etcdImageEnvVar); imgFromEnv != "" {
img = imgFromEnv
}

code := m.Run()
ctx := context.Background()

Comment on lines +29 to 30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add a timeout-bound context to prevent hanging tests

context.Background() has an unlimited lifetime. If the container pull or startup hangs (for example, due to a flaky network or Docker daemon issue), the test will block indefinitely. Consider adding a reasonable timeout:

-ctx := context.Background()
+ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+defer cancel()

This keeps the test suite responsive in CI environments.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ctx := context.Background()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
🤖 Prompt for AI Agents (early access) 
In etcd/etcd_test.go around lines 29 to 30, replace the use of context.Background() with a context that has a timeout to prevent the test from hanging indefinitely. Create a context with a timeout using context.WithTimeout, specifying a reasonable duration (e.g., a few seconds), and ensure to defer the cancellation function to release resources. This change will keep the test suite responsive by automatically cancelling the context if the operation takes too long.

_ = testStore.Reset()
_ = testStore.Close()
os.Exit(code)
// create a 2-node cluster
c, err := etcd.Run(ctx, img, etcd.WithNodes("etcd-1", "etcd-2"), etcd.WithClusterToken("test-cluster"))
testcontainers.CleanupContainer(t, c)
require.NoError(t, err)

hostPort, err := c.ClientEndpoint(ctx)
require.NoError(t, err)

return New(Config{
Endpoints: []string{hostPort},
})
}

func TestSetEtcd_ShouldReturnNoError(t *testing.T) {
Expand All @@ -28,18 +47,27 @@ func TestSetEtcd_ShouldReturnNoError(t *testing.T) {
val = []byte("doe")
)

testStore := newTestStore(t)
defer testStore.Close()

err := testStore.Set(key, val, 0)
require.NoError(t, err)
}

func TestGetEtcd_ShouldReturnNil_WhenDocumentNotFound(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

val, err := testStore.Get("not_found_key")

require.NoError(t, err)
require.Zero(t, len(val))
}

func TestSetAndGet_GetShouldReturn_SettedValueWithoutError(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

err := testStore.Set("test", []byte("fiber_test_value"), 0)
require.NoError(t, err)

Expand All @@ -50,6 +78,9 @@ func TestSetAndGet_GetShouldReturn_SettedValueWithoutError(t *testing.T) {
}

func TestSetAndGet_GetShouldReturnNil_WhenTTLExpired(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

err := testStore.Set("test", []byte("fiber_test_value"), 3*time.Second)
require.NoError(t, err)

Expand All @@ -62,6 +93,9 @@ func TestSetAndGet_GetShouldReturnNil_WhenTTLExpired(t *testing.T) {
}

func TestSetAndDelete_DeleteShouldReturn_NoError(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

err := testStore.Set("test", []byte("fiber_test_value"), 0)
require.NoError(t, err)

Expand All @@ -73,6 +107,9 @@ func TestSetAndDelete_DeleteShouldReturn_NoError(t *testing.T) {
}

func TestSetAndReset_ResetShouldReturn_NoError(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

err := testStore.Set("test", []byte("fiber_test_value"), 0)
require.NoError(t, err)

Expand All @@ -84,15 +121,22 @@ func TestSetAndReset_ResetShouldReturn_NoError(t *testing.T) {
}

func TestClose_CloseShouldReturn_NoError(t *testing.T) {
testStore := newTestStore(t)
err := testStore.Close()
require.NoError(t, err)
}

func TestGetConn_ReturnsNotNill(t *testing.T) {
testStore := newTestStore(t)
defer testStore.Close()

require.True(t, testStore.Conn() != nil)
}

func Benchmark_Etcd_Set(b *testing.B) {
testStore := newTestStore(b)
defer testStore.Close()

b.ReportAllocs()
b.ResetTimer()

Expand All @@ -105,6 +149,9 @@ func Benchmark_Etcd_Set(b *testing.B) {
}

func Benchmark_Etcd_Get(b *testing.B) {
testStore := newTestStore(b)
defer testStore.Close()

err := testStore.Set("john", []byte("doe"), 0)
require.NoError(b, err)

Expand All @@ -119,6 +166,9 @@ func Benchmark_Etcd_Get(b *testing.B) {
}

func Benchmark_Etcd_SetAndDelete(b *testing.B) {
testStore := newTestStore(b)
defer testStore.Close()

b.ReportAllocs()
b.ResetTimer()

Expand Down
77 changes: 61 additions & 16 deletions etcd/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,75 @@
module github.com/gofiber/storage/etcd/v2

go 1.19
go 1.23.0

require (
github.com/stretchr/testify v1.10.0
go.etcd.io/etcd/client/v3 v3.5.12
github.com/testcontainers/testcontainers-go v0.37.0
github.com/testcontainers/testcontainers-go/modules/etcd v0.37.0
go.etcd.io/etcd/client/v3 v3.5.21
)

require (
github.com/coreos/go-semver v0.3.1 // indirect
dario.cat/mergo v1.0.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/coreos/go-semver v0.3.0 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/docker v28.0.1+incompatible // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/ebitengine/purego v0.8.2 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/sys/user v0.1.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
go.etcd.io/etcd/api/v3 v3.5.12 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.12 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.25.0 // indirect
golang.org/x/net v0.23.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
google.golang.org/genproto v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/shirou/gopsutil/v4 v4.25.1 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.etcd.io/etcd/api/v3 v3.5.21 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.21 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.6.0 // indirect
go.uber.org/zap v1.17.0 // indirect
golang.org/x/crypto v0.37.0 // indirect
golang.org/x/net v0.38.0 // indirect
golang.org/x/sys v0.32.0 // indirect
golang.org/x/text v0.24.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/grpc v1.70.0 // indirect
google.golang.org/protobuf v1.36.5 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading
Loading