Skip to content

Go: Enable Testing with RC #3916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ on:
description: "Run the full engine, host, and language version matrix"
type: boolean
default: false
rc-version:
required: false
type: string
description: "RC Version number (Format: vX.Y.Z or vX.Y.Z-rcN)"
name:
required: false
type: string
Expand Down Expand Up @@ -122,6 +126,14 @@ jobs:
${{ matrix.host.TARGET }}-glide-core
${{ matrix.host.TARGET }}

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
run: |
Expand Down Expand Up @@ -155,6 +167,7 @@ jobs:
lint:
timeout-minutes: 10
runs-on: ubuntu-latest
if: ${{ github.event.inputs.rc-test != 'true' }}
steps:
- uses: actions/checkout@v4

Expand All @@ -178,7 +191,6 @@ jobs:
restore-keys: |
x86_64-unknown-linux-gnu-glide-core
x86_64-unknown-linux-gnu

- name: lint rust
uses: ./.github/workflows/lint-rust
with:
Expand Down Expand Up @@ -254,8 +266,17 @@ jobs:
key: ${{ matrix.host.IMAGE }}-go
restore-keys: ${{ matrix.host.IMAGE }}

- name: Run patch for RC testing
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
if: ${{ github.event.inputs.rc-test != 'true' }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if: ${{ github.event.inputs.rc-test != 'true' }}

And this step will run tests (yes, build will be odd for RC testing)

run: |
make install-tools build
make -k unit-test integ-test
Expand Down Expand Up @@ -303,10 +324,18 @@ jobs:
- name: Install zig
uses: ./.github/workflows/install-zig

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Build and test
working-directory: ./go
run: |
make install-tools build modules-test cluster-endpoints=${{ secrets.MEMDB_MODULES_ENDPOINT }} tls=true
make install-tools modules-test cluster-endpoints=${{ secrets.MEMDB_MODULES_ENDPOINT }} tls=true

- name: Upload test reports
if: always()
Expand Down Expand Up @@ -361,6 +390,14 @@ jobs:
${{ matrix.host.TARGET }}-glide-core
${{ matrix.host.TARGET }}

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version && github.event.inputs.rc-version >= 'v2.0.0' }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
run: |
Expand Down
80 changes: 80 additions & 0 deletions go/scripts/rc-testing/gha.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
diff --git a/.github/workflows/go.yml b/.github/workflows/go.yml
index d83c6f577..6eb9046ba 100644
--- a/.github/workflows/go.yml
+++ b/.github/workflows/go.yml
@@ -137,8 +137,8 @@ jobs:
- name: Install & build & test
working-directory: go
run: |
- make install-tools build
- make -k unit-test integ-test
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we not running unit-tests?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember the exact error correctly, it was because I ran into errors like this

/usr/local/Cellar/go/1.24.3/libexec/pkg/tool/darwin_amd64/link: running cc failed: exit status 1
/usr/bin/cc -arch x86_64 -m64 -Wl,-S -Wl,-x -o $WORK/b243/integTest.test -Qunused-arguments /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/go.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000000.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000001.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000002.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000003.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000004.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000005.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000006.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000007.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000008.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000009.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000010.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000011.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000012.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000013.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000014.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000015.o /var/folders/c9/lnprlxdx4zlgfz2mkqtq2h4r0000gp/T/go-link-944871722/000016.o -lresolv -O2 -g -lglide_rs -lm -framework Security -O2 -g -O2 -g -lpthread -framework CoreFoundation -framework Security
ld: library 'glide_rs' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)

+ make install-tools
+ make -k integ-test
make -k pubsub-test
make -k opentelemetry-test

@@ -164,44 +164,6 @@ jobs:
benchmarks/results/**
go/reports/**

- lint:
- timeout-minutes: 10
- runs-on: ubuntu-latest
- if: ${{ github.event.inputs.rc-test != 'true' }}
- steps:
- - uses: actions/checkout@v4
-
- - name: Set up Go
- uses: actions/setup-go@v5
- with:
- go-version: ${{ env.BASE_GO_VERSION }}
- cache-dependency-path: go/go.sum
-
- - name: Install protoc
- uses: ./.github/workflows/install-rust-and-protoc
- with:
- github-token: ${{ secrets.GITHUB_TOKEN }}
-
- - uses: actions/cache@v4
- with:
- path: |
- ffi/target
- glide-core/src/generated
- key: x86_64-unknown-linux-gnu-go
- restore-keys: |
- x86_64-unknown-linux-gnu-glide-core
- x86_64-unknown-linux-gnu
- - name: lint rust
- uses: ./.github/workflows/lint-rust
- with:
- cargo-toml-folder: ffi
- github-token: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Install and run linters
- working-directory: go
- run: |
- make install-dev-tools install-build-tools gen-c-bindings generate-protobuf lint-ci
-
get-containers:
runs-on: ubuntu-latest
if: ${{ github.event.inputs.full-matrix == 'true' || github.event_name == 'schedule' }}
@@ -278,8 +240,8 @@ jobs:
working-directory: go
if: ${{ github.event.inputs.rc-test != 'true' }}
run: |
- make install-tools build
- make -k unit-test integ-test
+ make install-tools
+ make -k integ-test
make -k pubsub-test

- name: Upload test reports
@@ -401,7 +363,7 @@ jobs:
- name: Install & build & test
working-directory: go
run: |
- make install-tools build
+ make install-tools
make -k long-timeout-test test-filter=TestLongTimeout

- name: Upload logs and reports
73 changes: 73 additions & 0 deletions go/scripts/rc-testing/rc-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#!/bin/bash
set -e

# Start by patching
cd ../
git apply -v go/scripts/rc-testing/gha.patch
cd go

# Check if version parameter is provided
if [ $# -ne 1 ]; then
echo "Usage: $0 <version>"
echo "Example: $0 v1.3.4-rc2"
exit 1
fi

VERSION=$1

# Validate version format
if ! [[ $VERSION =~ ^v([0-9]+)\.([0-9]+)\.([0-9]+)(-rc[0-9]+)?$ ]]; then
echo "Error: Version must be in format vX.Y.Z or vX.Y.Z-rcN (e.g., v1.3.4-rc2)"
exit 1
else
MAJOR=${BASH_REMATCH[1]}
echo "Testing release candidate: $VERSION"
fi

# Create testutils directory and copy internal interfaces to avoid internal package restrictions
echo "Setting up testutils to handle internal package imports..."
mkdir -p testutils
cp -r internal/interfaces/* testutils/

# Fix imports in test files to use testutils instead of internal/interfaces
echo "Updating test file imports..."
find integTest/ -name "*.go" -type f -exec grep -l "github.com/valkey-io/valkey-glide/go/v2/internal/interfaces" {} \; | while read -r file; do
echo "Updating imports in $file"
os="$(uname -s)"
if [ "$os" = "Linux" ]; then
sed -i 's|github.com/valkey-io/valkey-glide/go/v2/internal/interfaces|github.com/valkey-io/valkey-glide/go-test-rc/testutils|g' "$file"
else
# MacOS
sed -i '' 's|github.com/valkey-io/valkey-glide/go/v2/internal/interfaces|github.com/valkey-io/valkey-glide/go-test-rc/testutils|g' "$file"
fi
done

# Modify go.mod file
echo "Modifying go.mod file..."
# Use sed to modify the module name and add the dependency after testify depending on MacOS or Linux
os="$(uname -s)"
if [ "$os" = "Linux" ]; then
# Linux
sed -i -e 's|module github.com/valkey-io/valkey-glide/go/v2|module github.com/valkey-io/valkey-glide/go-test-rc|' go.mod
sed -i -e '/github.com\/stretchr\/testify/a\\tgithub.com/valkey-io/valkey-glide/go/v2 '"$VERSION" go.mod
# Add replace directive after the require block
sed -i -e '/^)/a\\nreplace github.com/valkey-io/valkey-glide/go/v2 => ./' go.mod
if [ $MAJOR -lt "2" ]; then # Fix for release version 1 branches
sed -i -e 's|"redis-cli",|CLI_COMMAND,|g' ../utils/cluster_manager.py
fi
else
# MacOS
sed -i '' -e 's|module github.com/valkey-io/valkey-glide/go/v2|module github.com/valkey-io/valkey-glide/go-test-rc|' go.mod
sed -i '' -e '/github.com\/stretchr\/testify/a\
github.com/valkey-io/valkey-glide/go/v2 '"$VERSION" go.mod
# Add replace directive after the require block
sed -i '' -e '/^)/a\
\
replace github.com/valkey-io/valkey-glide/go/v2 => ./' go.mod
if [ $MAJOR -lt "2" ]; then # Fix for release version 1 branches
sed -i '' -e 's|"redis-cli",|CLI_COMMAND,|g' ../utils/cluster_manager.py
fi
fi

echo "go.mod has been updated"
go mod tidy
Loading