fix: validate Weaviate collection existence before trusting Redis cache

[majiayu000]

Summary

When the Redis cache key vector_indexing_{collection_name} is still alive but the Weaviate class has been deleted externally (container restart, manual cleanup, ephemeral storage), _create_collection() trusts the stale cache and skips schema creation. This causes Weaviate's auto-schema to infer wrong property types (uuid instead of text for doc_id, document_id), resulting in silent RAG retrieval failure — all documents show as "completed" but return 0 search results because vectors are silently dropped.

This PR adds a collection existence check when the Redis cache hits. If the Weaviate class no longer exists, the stale cache key is cleared and schema creation proceeds normally with the correct property types.

Fixes #32458

Changes

• Added self._client.collections.exists() validation in _create_collection() when Redis cache indicates the collection was already indexed
• If the collection no longer exists in Weaviate, the stale Redis cache key is deleted and a warning is logged before proceeding with normal collection creation

Test plan

• Existing integration test tests/integration_tests/vdb/weaviate/test_weaviate.py covers the Weaviate vector store operations
• The fix is a guard clause that adds an existence check to an existing code path — no new behavior is introduced, only validation of an assumption (cache hit → collection exists)
• Manual verification scenario: create a knowledge base with Weaviate, delete the Weaviate class externally while Redis TTL is active, trigger re-indexing — collection should be recreated with correct schema

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical issue where a stale Redis cache entry could lead to incorrect Weaviate schema creation if the corresponding collection was externally deleted. The fix introduces a robust validation step to confirm the Weaviate collection's existence when the cache indicates it's present. If the collection is not found, the stale cache is invalidated, preventing silent data retrieval failures caused by schema mismatches and ensuring data consistency.

Highlights

• Weaviate Collection Validation: Added self._client.collections.exists() validation within _create_collection() to verify the actual existence of a Weaviate collection, even if the Redis cache suggests it's present.
• Stale Cache Handling: Implemented logic to clear stale Redis cache keys and log a warning if a Weaviate collection is found to be missing despite a cache hit, ensuring normal schema creation proceeds.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• api/core/rag/datasource/vdb/weaviate/weaviate_vector.py
  • Introduced a check for Weaviate collection existence when a Redis cache hit occurs.
  • Added logic to delete stale Redis cache keys and log a warning if the Weaviate collection is not found.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-03-12 16:12:03.267220963 +0000
+++ /tmp/pyrefly_pr.txt	2026-03-12 16:11:54.569185147 +0000
@@ -385,7 +385,7 @@
 ERROR Object of class `list` has no attribute `fields` [missing-attribute]
    --> core/rag/datasource/vdb/vikingdb/vikingdb_vector.py:143:55
 ERROR Class member `WeaviateVector._get_uuids` overrides parent class `BaseVector` in an inconsistent manner [bad-param-name-override]
-   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:237:9
+   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:243:9
 ERROR `response` may be uninitialized [unbound-name]
    --> core/rag/extractor/firecrawl/firecrawl_app.py:134:16
 ERROR `response` may be uninitialized [unbound-name]

[gemini-code-assist]

Code Review

This pull request addresses a critical bug where a stale Redis cache could lead to silent data indexing failures in Weaviate. The proposed solution, which validates the existence of a Weaviate collection even on a cache hit, is sound and directly tackles the root cause. My review includes one suggestion to improve efficiency by avoiding a redundant check that is introduced in the new logic for handling stale cache entries.

[gemini-code-assist]

This change correctly handles stale cache entries. However, in the case of a stale cache (when self._client.collections.exists returns False), the code proceeds to the try block below, where self._client.collections.exists() is called again on line 189. This results in a redundant network call.

Since this happens while holding a Redis lock, it unnecessarily increases the duration the lock is held and could increase lock contention. Consider refactoring to avoid this second check, for instance, by storing the result of the first exists() call in a variable and reusing it.

[github-actions]

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-03-12 16:38:51.468976828 +0000
+++ /tmp/pyrefly_pr.txt	2026-03-12 16:38:41.329860508 +0000
@@ -385,7 +385,7 @@
 ERROR Object of class `list` has no attribute `fields` [missing-attribute]
    --> core/rag/datasource/vdb/vikingdb/vikingdb_vector.py:143:55
 ERROR Class member `WeaviateVector._get_uuids` overrides parent class `BaseVector` in an inconsistent manner [bad-param-name-override]
-   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:237:9
+   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:243:9
 ERROR `response` may be uninitialized [unbound-name]
    --> core/rag/extractor/firecrawl/firecrawl_app.py:134:16
 ERROR `response` may be uninitialized [unbound-name]

[github-actions]

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-03-13 01:59:05.751075287 +0000
+++ /tmp/pyrefly_pr.txt	2026-03-13 01:58:57.163994430 +0000
@@ -385,7 +385,7 @@
 ERROR Object of class `list` has no attribute `fields` [missing-attribute]
    --> core/rag/datasource/vdb/vikingdb/vikingdb_vector.py:143:55
 ERROR Class member `WeaviateVector._get_uuids` overrides parent class `BaseVector` in an inconsistent manner [bad-param-name-override]
-   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:237:9
+   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:249:9
 ERROR `response` may be uninitialized [unbound-name]
    --> core/rag/extractor/firecrawl/firecrawl_app.py:134:16
 ERROR `response` may be uninitialized [unbound-name]

[github-actions]

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-03-13 15:37:07.663472050 +0000
+++ /tmp/pyrefly_pr.txt	2026-03-13 15:36:58.526402790 +0000
@@ -385,7 +385,7 @@
 ERROR Object of class `list` has no attribute `fields` [missing-attribute]
    --> core/rag/datasource/vdb/vikingdb/vikingdb_vector.py:143:55
 ERROR Class member `WeaviateVector._get_uuids` overrides parent class `BaseVector` in an inconsistent manner [bad-param-name-override]
-   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:237:9
+   --> core/rag/datasource/vdb/weaviate/weaviate_vector.py:249:9
 ERROR `response` may be uninitialized [unbound-name]
    --> core/rag/extractor/firecrawl/firecrawl_app.py:134:16
 ERROR `response` may be uninitialized [unbound-name]