Skip to content

Go: Enable Testing with RC #3916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ on:
description: "Run the full engine, host, and language version matrix"
type: boolean
default: false
rc-version:
required: false
type: string
description: "RC Version number (Format: vX.Y.Z or vX.Y.Z-rcN)"
name:
required: false
type: string
Expand Down Expand Up @@ -118,6 +122,14 @@ jobs:
${{ matrix.host.TARGET }}-glide-core
${{ matrix.host.TARGET }}

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
run: |
Expand Down Expand Up @@ -150,6 +162,7 @@ jobs:
lint:
timeout-minutes: 10
runs-on: ubuntu-latest
if: ${{ github.event.inputs.rc-test != 'true' }}
steps:
- uses: actions/checkout@v4

Expand All @@ -173,7 +186,6 @@ jobs:
restore-keys: |
x86_64-unknown-linux-gnu-glide-core
x86_64-unknown-linux-gnu

- name: lint rust
uses: ./.github/workflows/lint-rust
with:
Expand Down Expand Up @@ -249,8 +261,17 @@ jobs:
key: ${{ matrix.host.IMAGE }}-go
restore-keys: ${{ matrix.host.IMAGE }}

- name: Run patch for RC testing
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
if: ${{ github.event.inputs.rc-test != 'true' }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if: ${{ github.event.inputs.rc-test != 'true' }}

And this step will run tests (yes, build will be odd for RC testing)

run: |
make install-tools build
make -k unit-test integ-test
Expand Down Expand Up @@ -298,10 +319,18 @@ jobs:
- name: Install zig
uses: ./.github/workflows/install-zig

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Build and test
working-directory: ./go
run: |
make install-tools build modules-test cluster-endpoints=${{ secrets.MEMDB_MODULES_ENDPOINT }} tls=true
make install-tools modules-test cluster-endpoints=${{ secrets.MEMDB_MODULES_ENDPOINT }} tls=true

- name: Upload test reports
if: always()
Expand Down Expand Up @@ -356,6 +385,14 @@ jobs:
${{ matrix.host.TARGET }}-glide-core
${{ matrix.host.TARGET }}

- name: Run patch to test with RC
working-directory: go
if: ${{ github.event.inputs.rc-version && github.event.inputs.rc-version >= 'v2.0.0' }}
env:
RC_VERSION: ${{ github.event.inputs.rc-version }}
run: |
./scripts/rc-testing/rc-test.sh "$RC_VERSION"

- name: Install & build & test
working-directory: go
run: |
Expand Down
80 changes: 80 additions & 0 deletions go/scripts/rc-testing/gha.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
diff --git a/.github/workflows/go.yml b/.github/workflows/go.yml
index e76a34a3a..2641d9890 100644
--- a/.github/workflows/go.yml
+++ b/.github/workflows/go.yml
@@ -133,8 +133,8 @@ jobs:
- name: Install & build & test
working-directory: go
run: |
- make install-tools build
- make -k unit-test integ-test
+ make install-tools
+ make -k integ-test
make -k pubsub-test

- name: Run Example Tests
@@ -159,44 +159,6 @@ jobs:
benchmarks/results/**
go/reports/**

- lint:
- timeout-minutes: 10
- runs-on: ubuntu-latest
- if: ${{ github.event.inputs.rc-test != 'true' }}
- steps:
- - uses: actions/checkout@v4
-
- - name: Set up Go
- uses: actions/setup-go@v5
- with:
- go-version: ${{ env.BASE_GO_VERSION }}
- cache-dependency-path: go/go.sum
-
- - name: Install protoc
- uses: ./.github/workflows/install-rust-and-protoc
- with:
- github-token: ${{ secrets.GITHUB_TOKEN }}
-
- - uses: actions/cache@v4
- with:
- path: |
- ffi/target
- glide-core/src/generated
- key: x86_64-unknown-linux-gnu-go
- restore-keys: |
- x86_64-unknown-linux-gnu-glide-core
- x86_64-unknown-linux-gnu
- - name: lint rust
- uses: ./.github/workflows/lint-rust
- with:
- cargo-toml-folder: ffi
- github-token: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Install and run linters
- working-directory: go
- run: |
- make install-dev-tools install-build-tools gen-c-bindings generate-protobuf lint-ci
-
get-containers:
runs-on: ubuntu-latest
if: ${{ github.event.inputs.full-matrix == 'true' || github.event_name == 'schedule' }}
@@ -273,8 +235,8 @@ jobs:
working-directory: go
if: ${{ github.event.inputs.rc-test != 'true' }}
run: |
- make install-tools build
- make -k unit-test integ-test
+ make install-tools
+ make -k integ-test
make -k pubsub-test

- name: Upload test reports
@@ -396,7 +358,7 @@ jobs:
- name: Install & build & test
working-directory: go
run: |
- make install-tools build
+ make install-tools
make -k long-timeout-test test-filter=TestLongTimeout

- name: Upload logs and reports
58 changes: 58 additions & 0 deletions go/scripts/rc-testing/rc-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/bin/bash
set -e

# Start by patching
cd ../
git apply -v go/scripts/rc-testing/gha.patch
cd go

# Check if version parameter is provided
if [ $# -ne 1 ]; then
echo "Usage: $0 <version>"
echo "Example: $0 v1.3.4-rc2"
exit 1
fi

VERSION=$1

# Validate version format
if ! [[ $VERSION =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-rc[0-9]+)?$ ]]; then
echo "Error: Version must be in format vX.Y.Z or vX.Y.Z-rcN (e.g., v1.3.4-rc2)"
exit 1
else
MAJOR=${BASH_REMATCH[1]}
MINOR=${BASH_REMATCH[2]}
BRANCH="release-${MAJOR}.${MINOR}"

echo "Testing release candidate: $VERSION"
echo "Determined branch: $BRANCH"

# Checkout release branch
echo "Checking out $BRANCH branch..."
git fetch origin $BRANCH
git checkout $BRANCH
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to checkout a branch. GHA already has a branch selector, but we can run new tests with an old RC with your script.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be required. Go mod tidy doesn't seem to work too well when things have been moved around or added. Adding the branch makes it more stable

Ex. trying to run 1.3.4-rc2 on more recent branches

go.mod has been updated
go: finding module for package github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson
go: finding module for package github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson/options
go: github.com/valkey-io/valkey-glide/go-test-rc/api/server-modules/glidejson imports
        github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson/options: module github.com/valkey-io/valkey-glide/go@latest found (v1.3.4), but does not contain package github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson/options
go: github.com/valkey-io/valkey-glide/go-test-rc/integTest tested by
        github.com/valkey-io/valkey-glide/go-test-rc/integTest.test imports
        github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson: module github.com/valkey-io/valkey-glide/go@latest found (v1.3.4), but does not contain package github.com/valkey-io/valkey-glide/go/api/server-modules/glidejson

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but then again if we were to apply the patch and then switch to another release branch, conflicts start happening 😨

fi

# Modify go.mod file
echo "Modifying go.mod file..."
# Use sed to modify the module name and add the dependency after testify depending on MacOS or Linux
os="$(uname -s)"
if [ "$os" = "Linux" ]; then
# Linux
sed -i -e 's|module github.com/valkey-io/valkey-glide/go|module github.com/valkey-io/valkey-glide/go-test-rc|' go.mod
sed -i -e '/github.com\/stretchr\/testify/a\\tgithub.com/valkey-io/valkey-glide/go '"$VERSION"'' go.mod
if [ $MAJOR -lt "2" ]; then # Fix for release version 1 branches
sed -i -e 's|"redis-cli",|CLI_COMMAND,|g' ../utils/cluster_manager.py
fi
else
# MacOS
sed -i '' -e 's|module github.com/valkey-io/valkey-glide/go|module github.com/valkey-io/valkey-glide/go-test-rc|' go.mod
sed -i '' -e '/github.com\/stretchr\/testify/ a\
github.com\/valkey-io\/valkey-glide\/go '"$VERSION"'' go.mod
if [ $MAJOR -lt "2" ]; then # Fix for release version 1 branches
sed -i '' -e 's|"redis-cli",|CLI_COMMAND,|g' ../utils/cluster_manager.py
fi
fi

echo "go.mod has been updated"
go mod tidy
Loading