fix: skip reference instructions in LLM prompt when include_references=false#2833
fix: skip reference instructions in LLM prompt when include_references=false#2833majiayu000 wants to merge 2 commits intoHKUDS:mainfrom
Conversation
…s=false When include_references=false, the LLM prompt still contained reference formatting instructions (sections 4-5, reference_id tracking steps) and the context included the Reference Document List. This caused the LLM to generate reference markers in response text even though the API stripped the structured references field. Add no-reference prompt variants that omit reference instructions and context templates that omit Reference Document List. Conditionally select these variants in kg_query, _build_context_str, and naive_query when include_references is disabled. Closes HKUDS#2832 Signed-off-by: majiayu000 <1835304752@qq.com>
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6bd724483
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| sys_prompt_temp = system_prompt if system_prompt else PROMPTS["rag_response"] | ||
| if system_prompt: | ||
| sys_prompt_temp = system_prompt | ||
| elif not query_param.include_references: |
There was a problem hiding this comment.
This branch makes kg_query generate a different prompt when include_references is false, but the query cache key still omits that flag in the subsequent compute_args_hash(...) call. With enable_llm_cache=true, a response cached for one setting can be served to the other (e.g., cached cited answer returned when citations are disabled), violating the API contract for include_references.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Already fixed in f956920 — added include_references to the compute_args_hash call for kg_query.
There was a problem hiding this comment.
Already fixed in f956920 — added include_references to the compute_args_hash call for kg_query.
When enable_llm_cache=true, kg_query and naive_query could return stale cached responses generated under the opposite include_references setting. Add include_references to compute_args_hash inputs for both functions so the cache distinguishes between the two modes. Signed-off-by: majiayu000 <1835304752@qq.com>
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2 participants
Description
When
include_references=False, the structuredreferencesfield is correctly nulled, but the LLM system prompt still contains reference formatting instructions (steps 1.4-1.6, section 4, section 5) and the context still includes the "Reference Document List" section. This causes the LLM to generate reference markers in the response text even though the API strips the structured references field.Related Issues
Fixes #2832
Changes Made
lightrag/prompt.py— Added no-reference variants of prompt templates:rag_response_no_ref:rag_responsewithout reference tracking instructions, References Section Format, and Reference Section Examplenaive_rag_response_no_ref: Same treatment fornaive_rag_responsekg_query_context_no_ref:kg_query_contextwithout "Reference Document List" block and reference_id mentions in Document Chunks headernaive_query_context_no_ref: Same treatment fornaive_query_contextlightrag/operate.py— Conditional prompt selection:_build_context_str: Selects no-ref system prompt and context template wheninclude_references=False; skips buildingreference_list_strkg_query: Selects no-ref system prompt wheninclude_references=Falseand no customsystem_promptprovidednaive_rag: Same treatment for naive query modetests/test_include_references_prompt.py— 10 offline tests verifying:Checklist
Additional Notes
All 10 new tests pass. Pre-existing test failures in
test_postgres_index_name.pyare unrelated (postgres-specific tests that fail without asyncpg installed).