Skip to content

Python SDK: Redis vector store tests false-pass their skip check on vanilla Redis, then hard-fail #6686

Description

@ratishjain12

Summary

TestRedisThreshold (tests/vector_stores/test_e2e_threshold.py) and
TestRedis (tests/vector_stores/test_score_normalization.py) are meant to
skip when Redis isn't available, per the file's own docstring: "External
providers (PGVector, Redis, Milvus, etc.) are skipped unless the service is
reachable." Their skip condition only checks TCP reachability, not whether
the connected Redis actually supports RediSearch (the FT.* command family
mem0's Redis vector store depends on). Anyone running a plain, non-Stack
Redis locally (e.g. brew install redis, common for caching/sessions/other
projects) gets a false pass on the skip check and a hard test failure
instead of a clean skip.

Steps to Reproduce

# Start a vanilla (non-Stack) Redis locally
redis-server --daemonize yes
redis-cli ping        # PONG
redis-cli MODULE LIST # (empty — no RediSearch loaded)

# Run the affected tests
pytest tests/vector_stores/test_e2e_threshold.py::TestRedisThreshold
pytest tests/vector_stores/test_score_normalization.py::TestRedis

### Expected Behavior
Per the file's own docstring, these tests should skip cleanly with a message
like "Redis not reachable" (or, more accurately, "RediSearch not available")
when the environment doesn't support what the test needs.

### Actual Behavior
Both tests run (skip condition wrongly evaluates to "don't skip") and then
fail with a raw client error:

FAILED tests/vector_stores/test_e2e_threshold.py::TestRedisThreshold::test_threshold_filtering
redis.exceptions.ResponseError: unknown command 'FT._LIST', with args beginning with:

ERROR tests/vector_stores/test_score_normalization.py::TestRedis::test_scores_are_similarity
redis.exceptions.ResponseError: unknown command 'FT._LIST', with args beginning with:

```bash
REDIS_HOST = os.environ.get("REDIS_HOST", "localhost")
REDIS_PORT = int(os.environ.get("REDIS_PORT", "6379"))

@pytest.mark.skipif(
    not _tcp_reachable(REDIS_HOST, REDIS_PORT),
    reason=f"Redis not reachable at {REDIS_HOST}:{REDIS_PORT}",
)

_tcp_reachable() only opens a socket to confirm something is listening —
it doesn't check what that something supports. RediSearch (FT.*) is an
optional Redis module (bundled in Redis Stack), not part of vanilla Redis,
so a plain redis-server passes the reachability check but can't run the
test.

This also appears untested in CI: no workflow under .github/workflows/
configures a Redis service for the test suite, so these tests are always
skipped there (_tcp_reachable returns False, nothing is listening) and
this gap has never been exercised end-to-end in CI — only locally, for
contributors who happen to already have Redis running.

Environment
mem0 version: 2.0.14 (pyproject.toml)
Python version: 3.11 (Redis behavior is not Python-version-specific)
OS: macOS (Darwin) — reproducible on any OS with a vanilla Redis install; Redis version 8.8.0, no modules loaded (confirmed via MODULE LIST)

Metadata

Metadata

Assignees

No one assigned

    Labels

    sdk-pythonPython SDK specificvector-storeVector store backends (Qdrant, PGVector, Redis, etc.)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions