Python: Intermittent “Could not find the data in the given documents” response when querying search index with Magnetic Orchestration

[Roopan-Microsoft]

Describe the bug
We are using Magnetic Orchestration with Semantic Kernel where the RAG (Retrieval-Augmented Generation) approach is integrated with a search index to fetch results from documents.
Occasionally, the response returns:

Could not find the data in the given documents

Most of the time, relevant documents are retrieved successfully. We need to understand why sometimes no data is found in the index, even though the documents exist, and how to debug such cases.

Link: https://github.com/microsoft/Multi-Agent-Custom-Automation-Engine-Solution-Accelerator/blob/main/src/backend/v3/orchestration/orchestration_manager.py

To Reproduce
Steps to reproduce the behavior:
N/A

Expected behavior
Relevant documents should be consistently retrieved from the search index if they exist.
If no data is found, there should be clearer debug information/logging to identify the reason (e.g., query mismatch, index issue, embedding/vector similarity threshold, etc.).

Screenshots
N/A

Platform

• Language: Python
• Source: semantic-kernel==1.35.3
• AI model: gpt- 4.1-mini
• IDE: VS Code
• OS: Windows

Additional context

• Need clarification on possible reasons why the search index sometimes fails to return results.
• Need guidance on how to debug this issue
• Any best practices for monitoring/debugging RAG search failures with Semantic Kernel + Magnetic Orchestration would be helpful.

[Roopan-Microsoft]

Hey @moonbox3 @markwallace-microsoft - Any update on this?