Skip to content

test: add E2E tests for probe behavior - #11

Merged
usiegj00 merged 2 commits into
mainfrom
feature/e2e-probe-tests
Jan 25, 2026
Merged

test: add E2E tests for probe behavior#11
usiegj00 merged 2 commits into
mainfrom
feature/e2e-probe-tests

Conversation

@usiegj00

@usiegj00 usiegj00 commented Jan 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds E2E tests validating current probe behavior as a baseline before replacing process-spawning probes with instance manager HTTP endpoints.

Fixes #5

Test Coverage

Behavioral Tests (Implementation Agnostic)

  • Healthy pods remain Ready and Live
  • Pod restarts when Redis process dies (liveness probe)
  • Pod recovers after restart and becomes Ready
  • Data replicates from master to replicas (1000 keys)
  • Replica becomes Ready after resync (pod deletion + recreation)
  • Data survives pod restarts and failover
  • Sentinel pods are Ready
  • Redis cluster remains functional after all tests

Test Optimizations

  • Custom probe timings for faster tests:
    • Liveness: 5s initial, 5s period, 3 failures = ~20s to detect failure (vs 120s default)
    • Readiness: 5s initial, 3s period, 3 failures = ~14s to detect not-ready (vs 50s default)

Why This Matters

These tests validate outcomes, not mechanisms. When we swap the probes:

  • redis-cli ping/healthz HTTP endpoint
  • ready.sh script → /readyz HTTP endpoint

The same tests should pass. If they do, the swap was successful.

Test Plan

  • E2E workflow runs successfully
  • Pods stay Ready under normal operation
  • Liveness probe restart test passes
  • Replica resync test passes
  • Data survives failover
  • Redis cluster functional after tests

Add e2e-probe-behavior job that validates:
- Liveness probe configuration (redis-cli ping)
- Readiness probe configuration (ready.sh script)
- Liveness probe triggers pod restart on Redis failure
- Master pod is Ready
- Synced replica pods are Ready
- Sentinel liveness probe configuration
- Redis cluster functionality after probe tests

Part of issue #5
- Use custom probe timings for faster tests (20s vs 120s for liveness)
- Write 1000 keys to test replication works
- Verify data replicates to replicas
- Test replica resync: delete replica, verify it becomes Ready
- Test data survives pod restarts/failover
- Verify Sentinels are Ready
- Final functionality check with new data
@usiegj00
usiegj00 merged commit 1e98408 into main Jan 25, 2026
12 checks passed
@usiegj00
usiegj00 deleted the feature/e2e-probe-tests branch January 25, 2026 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add E2E tests validating current probe behavior

1 participant