Skip to content

Commit 6396496

Browse files
authored
Replace Docker cluster with cluster_manager.py (#108)
* feat(ruby): replace Docker cluster with cluster_manager.py - Add Valkey::TestCluster class wrapping cluster_manager.py invocation - Update Helper::Cluster to use TestCluster for dynamic cluster management - Update Helper::Client with standalone server support via TestCluster - Update CI workflow to use Python 3.11 + native Valkey installation - Remove grokzen/redis-cluster Docker dependency - Add rantly gem for property-based testing - Fix hardcoded port 7000 reference in connection_options.rb This aligns the Ruby client's cluster testing infrastructure with other GLIDE clients (Python, Java, Go) by using the shared cluster_manager.py script from valkey-glide/utils/. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ci): build Valkey from source instead of using packages.valkey.io packages.valkey.io DNS resolution was failing in GitHub Actions. Build Valkey 8.0.0 from source with TLS support, matching the approach used by valkey-glide's install-engine action. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix: resolve RuboCop offenses in test cluster infrastructure - Use ENV.fetch with nil default instead of ENV[] (FetchEnvVar) - Use modifier unless for single-line conditional (IfUnlessModifier) - Add test_cluster.rb to Metrics/ClassLength and MethodLength exclusions Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * refactor: rename test:valkey to test:standalone, run both by default - Rename test/valkey/ directory to test/standalone/ - Update Rakefile: test:valkey → test:standalone - Default 'test' task now runs both test:standalone and test:cluster - Update CI workflow to use test:standalone Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * refactor(ruby): unify test structure with shared ValkeyTests modules - Rename test/standalone/ to test/valkey/ for shared modules - Convert test classes to ValkeyTests::* modules (like Lint::*) - Create new test/standalone/ with test classes that include modules - Update test/cluster/cluster_commands_test.rb to include ValkeyTests modules - Update test_helper.rb to load valkey shared test modules - Fix assert_not_nil -> refute_nil for Minitest compatibility Signed-off-by: Alex Rehnby-Martin <alexrema@amazon.com> Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): separate test classes for modules with custom setup methods - Split Lint modules with setup methods (GeoCommands, JsonCommands, ModuleCommands, VectorSearchCommands) into their own test classes - Fix test isolation issue where GeoCommands.setup created 'Sicily' key that leaked into other tests like test_del - Add ensure_otel_initialized to OpenTelemetry module setup to handle random test ordering - Add skip for test_randomkey in cluster mode (requires isolated db) Fixes standalone test failures caused by setup method conflicts when multiple Lint modules were combined into a single test class. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): add cluster-mode skips for cross-slot operations - Skip cross-slot operations (smove, sinter, sdiff, zunion, zinter, etc.) in cluster mode - Skip rename/renamenx in cluster mode (different hash slots) - Skip lmove when using different keys across hash slots - Skip EXEC/DISCARD without MULTI in cluster mode - Fix function tests to clean up libraries before loading (rolib, policylib) - Relax OpenTelemetry span count assertions to handle test ordering Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * chore: remove spec files and notes from git tracking Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * refactor(ruby): restore _test suffix to test/valkey module files Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * chore(ruby): exclude test/valkey and test/lint from lost_tests check These directories contain reusable test modules (not standalone test files) that are included by test classes in test/standalone/ and test/cluster/. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): add cluster-mode skips for MULTI/EXEC and cross-slot tests - Skip all MULTI/EXEC transaction tests in cluster mode (connection routing issues) - Skip geosearchstore test (source/destination keys may be on different slots) - Skip sorting tests with GET/STORE (cross-slot operations) - Skip eval/evalsha tests with random keys (cross-slot operations) - Skip large_parameter_arrays test (50 keys on different slots) Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): add flush delay in OpenTelemetry test setup Wait 200ms before cleaning test files to allow any buffered spans from previous tests to flush. This fixes flaky span count assertions caused by async span flushing with 100ms flush interval. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): add cluster-mode skips for CrossSlot and destructive tests - Skip test_del, test_del_with_array_argument (untagged keys) - Skip test_scan (may not see all keys in cluster mode) - Skip test_select_database (behavior varies in cluster) - Skip test_memory_malloc_stats (returns multi-node Array) - Skip test_pfmerge (untagged keys foo/bar/res) - Skip test_script_execution_consistency, test_parameter_round_trip_preservation (random keys may be on different slots) - Skip destructive cluster slot management tests (addslotsrange, delslotsrange, addslots, delslots, setslot) to prevent cluster instability - Fix statistics tests to use _new_client helper instead of hard-coded port 7000 Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * style(ruby): fix rubocop offenses in test files - Convert rescue modifier to begin/rescue blocks in function_commands.rb - Fix comment annotation format in commands_test.rb Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): use _new_client helper in test_client_kill tests The tests were using Valkey.new which defaults to 127.0.0.1:6379, but in cluster mode there's no standalone server at that address. Use _new_client helper to get a properly configured client. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * fix(ruby): skip CLIENT KILL tests in cluster mode CLIENT KILL by address doesn't work reliably in cluster mode because the command may be routed to a different node than where the client is connected, resulting in 'No such client' errors. Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * Address PR feedback Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> * chore(ruby): address PR review feedback - Remove unused rantly gem from Gemfile - Remove unused start_server/stop_server methods from Helper::Client - Move test_cluster.rb from lib/ to test/support/ (not shipped in gem) - Update docs to use test:standalone instead of test:valkey - Update cd.yml to use rake test:standalone - Update .rubocop.yml to remove stale exclusions Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> --------- Signed-off-by: Alex Rehnby-Martin <alex.rehnby-martin@improving.com> Signed-off-by: Alex Rehnby-Martin <alexrema@amazon.com>
1 parent 1465b38 commit 6396496

56 files changed

Lines changed: 2235 additions & 1974 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/CI.yml

Lines changed: 24 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -225,7 +225,7 @@ jobs:
225225
bundler-cache: true
226226

227227
- name: Run standalone tests
228-
run: bundle exec rake test:valkey
228+
run: bundle exec rake test:standalone
229229

230230
- name: Stop Valkey containers
231231
if: always()
@@ -248,27 +248,27 @@ jobs:
248248
- 3.1
249249
- 3.0
250250

251-
services:
252-
valkey-cluster:
253-
image: grokzen/redis-cluster:7.0.15
254-
ports:
255-
- 7000:7000
256-
- 7001:7001
257-
- 7002:7002
258-
- 7003:7003
259-
- 7004:7004
260-
- 7005:7005
261-
options: >-
262-
--health-cmd "redis-cli -p 7000 ping && redis-cli -p 7001 ping && redis-cli -p 7002 ping"
263-
--health-interval 15s
264-
--health-timeout 10s
265-
--health-retries 15
266-
267251
steps:
268252
- uses: actions/checkout@v4
269253
with:
270254
submodules: recursive
271255

256+
- name: Set up Python
257+
uses: actions/setup-python@v5
258+
with:
259+
python-version: '3.11'
260+
261+
- name: Build and install Valkey from source
262+
run: |
263+
echo "Building Valkey 8.0.0 from source..."
264+
git clone https://github.com/valkey-io/valkey.git
265+
cd valkey
266+
git checkout 8.0.0
267+
make BUILD_TLS=yes -j4
268+
sudo make install
269+
echo "Verifying Valkey installation..."
270+
valkey-server --version
271+
272272
- name: Download native library
273273
uses: actions/download-artifact@v4
274274
with:
@@ -281,59 +281,14 @@ jobs:
281281
ruby-version: ${{ matrix.ruby }}
282282
bundler-cache: true
283283

284-
- name: Configure cluster nodes
285-
run: |
286-
echo "=== Disabling protected mode on all nodes ==="
287-
for port in 7000 7001 7002 7003 7004 7005; do
288-
echo "Configuring port $port..."
289-
docker exec $(docker ps -q | head -n1) redis-cli -p $port config set protected-mode no || echo "Failed to configure port $port"
290-
done
291-
292-
- name: Wait for cluster to be ready
293-
run: |
294-
echo "=== Waiting for Redis cluster to be fully ready ==="
295-
max_attempts=60
296-
attempt=1
297-
298-
while [ $attempt -le $max_attempts ]; do
299-
echo "Attempt $attempt/$max_attempts: Checking cluster state..."
300-
301-
# Check if all nodes are responding
302-
all_nodes_up=true
303-
for port in 7000 7001 7002 7003 7004 7005; do
304-
if ! docker exec $(docker ps -q | head -n1) redis-cli -p $port ping > /dev/null 2>&1; then
305-
echo " Node $port is not responding"
306-
all_nodes_up=false
307-
break
308-
fi
309-
done
310-
311-
if [ "$all_nodes_up" = true ]; then
312-
# Check cluster state
313-
cluster_state=$(docker exec $(docker ps -q | head -n1) redis-cli -p 7000 cluster info | grep "cluster_state" | cut -d: -f2 | tr -d '\r')
314-
echo " Cluster state: $cluster_state"
315-
316-
if [ "$cluster_state" = "ok" ]; then
317-
echo "✅ Cluster is ready!"
318-
break
319-
fi
320-
fi
321-
322-
echo " Cluster not ready yet, waiting 5 seconds..."
323-
sleep 5
324-
attempt=$((attempt + 1))
325-
done
326-
327-
if [ $attempt -gt $max_attempts ]; then
328-
echo "❌ Cluster failed to become ready within timeout"
329-
echo "=== Final cluster info ==="
330-
docker exec $(docker ps -q | head -n1) redis-cli -p 7000 cluster info || echo "Failed to get cluster info"
331-
echo "=== Cluster nodes ==="
332-
docker exec $(docker ps -q | head -n1) redis-cli -p 7000 cluster nodes || echo "Failed to get cluster nodes"
333-
exit 1
334-
fi
335-
336284
- name: Run cluster tests
337285
run: bundle exec rake test:cluster
338286
timeout-minutes: 25
339287

288+
- name: Upload cluster logs on failure
289+
if: failure()
290+
uses: actions/upload-artifact@v4
291+
with:
292+
name: cluster-logs-${{ matrix.ruby }}
293+
path: valkey-glide/utils/clusters/
294+
retention-days: 7

.github/workflows/cd.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ jobs:
280280
# The installed gem has the native library, local lib doesn't
281281
rm -rf lib/
282282
# Run tests - they will use the installed gem
283-
rake test:valkey
283+
rake test:standalone
284284
285285
- name: Stop Valkey
286286
if: always()

AGENTS.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,12 @@ bin/setup # bundle install
5454
bundle exec rubocop
5555

5656
# Testing
57-
bundle exec rake test # standalone (default)
58-
bundle exec rake test:valkey # standalone explicitly
59-
bundle exec rake test:cluster # cluster (needs nodes 7000-7005)
57+
bundle exec rake test # standalone + cluster
58+
bundle exec rake test:standalone # standalone only
59+
bundle exec rake test:cluster # cluster only (needs nodes 7000-7005)
6060

6161
# Verbose / CI mode
62-
CI=1 bundle exec rake test:valkey
62+
CI=1 bundle exec rake test:standalone
6363

6464
# Console
6565
bundle exec bin/console
@@ -72,7 +72,7 @@ bundle exec bin/console
7272
bundle exec ruby test/valkey/string_commands_test.rb
7373

7474
# Run with custom port
75-
VALKEY_PORT=6379 TIMEOUT=10 bundle exec rake test:valkey
75+
VALKEY_PORT=6379 TIMEOUT=10 bundle exec rake test:standalone
7676

7777
# Load gem from lib/ without install
7878
RUBYOPT="-I$(pwd)/lib" ruby -r valkey -e 'p Valkey.new.ping'
@@ -82,7 +82,7 @@ RUBYOPT="-I$(pwd)/lib" ruby -r valkey -e 'p Valkey.new.ping'
8282

8383
| Suite | Server requirement |
8484
|-------|-------------------|
85-
| `test:valkey` | Standalone Valkey/Redis on `localhost:6379` (DB 15) |
85+
| `test:standalone` | Standalone Valkey/Redis on `localhost:6379` (DB 15) |
8686
| `test:cluster` | 6-node cluster on `127.0.0.1:7000``7005` |
8787
| SSL tests | TLS Valkey on port `6380` + certs in `test/fixtures/ssl/` |
8888
| Module tests | JSON, Bloom, Search modules loaded (see CI workflow) |
@@ -191,7 +191,7 @@ valkey-glide-ruby/
191191
## Quality Gates (Agent Checklist)
192192

193193
- [ ] `bundle exec rubocop` passes
194-
- [ ] `bundle exec rake test:valkey` passes (with Valkey running)
194+
- [ ] `bundle exec rake test:standalone` passes (with Valkey running)
195195
- [ ] `bundle exec rake test:cluster` passes (if cluster commands touched)
196196
- [ ] New commands have tests in `test/valkey/` and lint coverage where applicable
197197
- [ ] `RequestType` matches glide-core enum

CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,14 +27,14 @@ Valkey GLIDE Ruby (`valkey-rb`) is the official Ruby binding for Valkey and Redi
2727

2828
### Before Task Completion
2929

30-
- Tests cover the change and pass (`bundle exec rake test:valkey` at minimum)
30+
- Tests cover the change and pass (`bundle exec rake test:standalone` at minimum)
3131
- `bundle exec rubocop` passes for changed Ruby files
3232
- If FFI binary updated, smoke-test: `Valkey.new.ping` on target OS
3333

3434
### Before Push
3535

3636
- `git pull --rebase` on your base branch; resolve conflicts
37-
- Run tests for relevant scope (`test:valkey` and/or `test:cluster`)
37+
- Run tests for relevant scope (`test:standalone` and/or `test:cluster`)
3838

3939
### Before PR Creation/Update
4040

@@ -108,7 +108,7 @@ When working on a feature, read these paths first:
108108
```bash
109109
bin/setup
110110
bundle exec rubocop
111-
bundle exec rake test:valkey # needs localhost:6379
111+
bundle exec rake test:standalone # needs localhost:6379
112112
bundle exec rake test:cluster # needs cluster 7000-7005
113113
```
114114

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ For issues that affect the shared Rust core or other language clients, consider
3030
3. Run local checks (see [DEVELOPER.md](./DEVELOPER.md)):
3131
```bash
3232
bundle exec rubocop
33-
bundle exec rake test:valkey # standalone — requires Valkey on :6379
33+
bundle exec rake test:standalone # standalone — requires Valkey on :6379
3434
bundle exec rake test:cluster # if cluster-related — requires nodes :7000–:7005
3535
```
3636
4. Commit with **DCO sign-off** and **conventional commits**:

DEVELOPER.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ valkey-glide-ruby/
4747
│ └── cd.yml # Build and publish gem
4848
├── valkey.gemspec
4949
├── Gemfile
50-
└── Rakefile # test:valkey, test:cluster, native:build
50+
└── Rakefile # test:standalone, test:cluster, native:build
5151
```
5252

5353
## Prerequisites
@@ -211,13 +211,13 @@ docker run -d --name redis-cluster -p 7000-7005:7000-7005 grokzen/redis-cluster:
211211
# All default (standalone) tests
212212
bundle exec rake test
213213
# or
214-
bundle exec rake test:valkey
214+
bundle exec rake test:standalone
215215

216216
# Cluster tests only
217217
bundle exec rake test:cluster
218218

219219
# Verbose output (also enabled when CI=1)
220-
CI=1 bundle exec rake test:valkey
220+
CI=1 bundle exec rake test:standalone
221221
```
222222

223223
### SSL Tests
@@ -243,7 +243,7 @@ Load modules when starting Valkey if you run module-specific tests locally.
243243
### Environment Overrides
244244

245245
```bash
246-
VALKEY_PORT=6379 TIMEOUT=10 bundle exec rake test:valkey
246+
VALKEY_PORT=6379 TIMEOUT=10 bundle exec rake test:standalone
247247
```
248248

249249
## Linters

Rakefile

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ task native: "native:build"
111111
# =============================================================================
112112

113113
namespace :test do
114-
groups = %i[valkey cluster]
114+
groups = %i[standalone cluster]
115115
groups.each do |group|
116116
Rake::TestTask.new(group) do |t|
117117
t.libs << "test"
@@ -122,10 +122,15 @@ namespace :test do
122122
end
123123
end
124124

125-
lost_tests = Dir["test/**/*_test.rb"] - groups.map { |g| Dir["test/#{g}/**/*_test.rb"] }.flatten
125+
# Exclude module directories (valkey/, lint/) from lost_tests check
126+
# These contain reusable test modules, not standalone test files
127+
module_dirs = %w[valkey lint]
128+
lost_tests = Dir["test/**/*_test.rb"] -
129+
groups.map { |g| Dir["test/#{g}/**/*_test.rb"] }.flatten -
130+
module_dirs.map { |d| Dir["test/#{d}/**/*_test.rb"] }.flatten
126131
abort "The following test files are in no group:\n#{lost_tests.join("\n")}" unless lost_tests.empty?
127132
end
128133

129-
task test: ["test:valkey"]
134+
task test: ["test:standalone", "test:cluster"]
130135

131136
task default: :test

test/cluster/cluster_commands_test.rb

Lines changed: 71 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,77 @@
22

33
require "test_helper"
44

5-
class TestClusterCommandsOnClusters < Minitest::Test
5+
# Cluster test class for command modules WITHOUT custom setup methods
6+
# These modules can safely share a test class
7+
class TestClusterCommands < Minitest::Test
68
include Helper::Cluster
7-
# include Lint::StringCommands # Run string tests first (while cluster is healthy)
8-
# include Lint::ServerCommands # Server commands work in cluster mode
9-
include Lint::HashCommands # Hash commands work in cluster mode
10-
include Lint::StreamCommands # Stream commands work in cluster mode
11-
include Lint::ConnectionCommands # Run connection tests (cluster-aware)
12-
include Lint::PubSubCommands # Pub/Sub works in cluster mode
13-
include Lint::FunctionCommands # Function commands work in cluster mode (per-node)
14-
include Lint::ModuleCommands # Module commands work in cluster mode (per-node)
15-
include Lint::JsonCommands # JSON commands work in cluster mode (requires module on all nodes)
16-
include Lint::ClusterCommands # Run cluster commands second (after string tests)
17-
include Lint::ConnectionOptions # Connection options work in cluster mode
9+
10+
# Lint modules without conflicting setup methods
11+
include Lint::BitmapCommands
12+
include Lint::ConnectionCommands
13+
include Lint::ConnectionOptions
14+
include Lint::FunctionCommands
15+
include Lint::GenericCommands
16+
include Lint::HashCommands
17+
include Lint::HyperLogLog
18+
include Lint::Lists
19+
include Lint::PubSubCommands
20+
include Lint::ScriptingCommands
21+
include Lint::ServerCommands
22+
include Lint::SetCommands
23+
include Lint::SortedSetCommands
24+
include Lint::StreamCommands
25+
include Lint::StringCommands
26+
include Lint::TransactionCommands
27+
28+
# Cluster-specific commands
29+
include Lint::ClusterCommands
30+
31+
# ValkeyTests modules without conflicting setup methods
32+
include ValkeyTests::Bitpos
33+
include ValkeyTests::GenericCommands
34+
include ValkeyTests::Scanning
35+
include ValkeyTests::ScriptingCommands
36+
include ValkeyTests::ScriptingCommandsIntegration
37+
include ValkeyTests::Sorting
38+
include ValkeyTests::Statistics
39+
include ValkeyTests::URIConnection
40+
include ValkeyTests::Utils
41+
42+
# Eval/Evalsha property tests
43+
include ValkeyTests::EvalEvalshaBasicProperties
44+
include ValkeyTests::EvalEvalshaValidationProperties
45+
include ValkeyTests::EvalEvalshaTypeProperties
46+
end
47+
48+
# Modules WITH custom setup methods need their own test class
49+
# to avoid setup interference between different command groups
50+
51+
class TestClusterGeoCommands < Minitest::Test
52+
include Helper::Cluster
53+
include Lint::GeoCommands
54+
end
55+
56+
class TestClusterJsonCommands < Minitest::Test
57+
include Helper::Cluster
58+
include Lint::JsonCommands
59+
end
60+
61+
class TestClusterModuleCommands < Minitest::Test
62+
include Helper::Cluster
63+
include Lint::ModuleCommands
64+
end
65+
66+
class TestClusterVectorSearchCommands < Minitest::Test
67+
include Helper::Cluster
68+
include Lint::VectorSearchCommands
69+
end
70+
71+
# ValkeyTests modules with setup/teardown
72+
class TestClusterFunctionCommands < Minitest::Test
73+
include Helper::Cluster
74+
include ValkeyTests::FunctionCommands
1875
end
1976

20-
# TODO: Enable module and load module in cluster setup CI
77+
# NOTE: Module tests (JsonCommands, ModuleCommands, VectorSearchCommands)
78+
# require modules loaded on all cluster nodes

0 commit comments

Comments
 (0)