[Performance] MiniRAG underperforms NaiveRAG significantly with Phi-3.5-mini（MiniRAG 在 Phi-3.5-mini 模型上的表现显著不如 NaiveRAG） #109
Description
Hello, I am attempting to reproduce the MiniRAG experiments using the LiHua-World dataset. I am using Microsoft/Phi-3.5-mini-instruct (deployed via vLLM) for both Indexing (Graph Construction) and Retrieval/QA.
Contrary to the findings in the paper, my experimental results show that the accuracy of NaiveRAG is significantly higher than that of MiniRAG.
In my latest experiments, with the same configuration:
param = QueryParam( # mode="naive", mode="mini", top_k=5, max_token_for_text_unit=2000, max_token_for_text=2000, max_token_for_global_context=2000, max_token_for_local_context=2000, )
NaiveRAG achieved an accuracy of 52.90%, while MiniRAG's accuracy was even below 40%.
In many other experimental groups (such as using the default max_token=4000), NaiveRAG's accuracy remained above 50%, whereas MiniRAG performed poorly.
Why is this?