feat(chat): add optional character RAG with bilingual starter knowledge - #59
Merged
Conversation
Lucas1479
added a commit
that referenced
this pull request
Sep 7, 2026
…jects (#60) The old 148-entry local corpus contains useful character facts alongside duplicates, incorrect member numbers and unverified anecdotes. This data-only follow-up to #59 audits every legacy row and expands the public corpus from **30 to 52 Chinese/Japanese records, covering 26 topics**. The update corrects Moeka/Ruka/Faris to 005/006/007, adds the Science publication, chopsticks/cooking, father relationship, PhoneWave, Okabe's phrase, supporting characters and the lab building, and deduplicates already covered facts. Twenty old rows have core facts already covered; verified portions of 16 more are incorporated/corrected; 112 are deferred for sourcing or narrative context, not declared universally false. The original local files remain unchanged. Sources and all row dispositions are documented. The father entry uses the maintainer's suggested subject anchoring: `私(牧瀬紅莉栖)の父親はドクター中鉢。` Other reviewed wording stays intact. A blanket first-person rewrite was tested but added four unrelated-query false hits. No runtime code, static persona, RAG wrapper, default-off setting, threshold, top-k, or dependencies change. Validation: - 70 targeted Python tests passed, including all 67 development retrieval cases and release-tooling tests; Ruff, architecture views and source-release validation passed. - Additional held-out retrieval improved from 2/18 with the 30-entry corpus to 11/18; remaining failures and six pre-existing unrelated-query false hits are explicitly recorded. - 153 live Flash requests across corpus comparison and follow-up wording tests. Central facts improved, but these are not 153 persona passes. RAG-off and first-person arms both produced an explicit `俺` in the focused experiment; shared-history fabrication and unsupported elaboration remain. [Audit, method, results and limitations](https://github.com/Code-Amadeus/Amadeus/blob/codex/character-knowledge-expansion/docs/character_rag_curation.md). Related to #56; this does not close its default-off persona acceptance criterion. --- 这是 #59 后续的纯资料更新:逐条审看旧本地 148 条,去重、核对及修正后,公开库从 **30 条扩为中日文 52 条、26 个主题**。20 条的核心事实已有覆盖,16 条中的可核实事实融入,112 条待来源或剧情上下文确认;不代表判定它们全部错误,旧本地文件未改动。 修正萌郁/琉华/菲利斯的编号,补充论文期刊、筷子与料理、父亲、电话微波炉、角色及场所等事实。父亲条目采用明确的“私(牧瀬紅莉栖)の父親は…”写法;全库统一改第一人称新增了四个无关问题误召回,因此没有直接采用。默认 prompt、RAG 包装、开关、阈值、top-k 和依赖不变。 70 项相关 Python 测试通过,包含 67 条真实检索开发回归;额外问法从 2/18 提升至 11/18,失败情况完整保留。另做 153 次真实模型调用,确认事实覆盖改善,但男性口吻、虚构共同经历及附加解释仍有反例,不宣称人物表现全部修好。建议作为默认关闭 RAG 的资料更新审阅,#56 继续开启。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main Chat can miss Kurisu's canonical BBS handle, especially across Chinese/Japanese spellings. This adds optional local character retrieval shared by all eight Chat routes, with a sourced bilingual starter corpus. RAG is off by default and the static persona is unchanged. Based on current public main
d5e1d7f, including #55..envor Settings. Show applied configuration, loading/failure state and last retrieval summary. CLI diagnostics show rejected candidates as well as accepted hits.ragextra without changing CPU/cu124/ROCm build selection; add a dedicated CPU retrieval CI job. Update both READMEs, setup/evaluation docs and source-archive selection.Validation: 107 targeted Python tests passed (1 skipped); all 32 real E5 retrieval development cases passed; 15 Electron tests and TypeScript/Vite build passed. Ruff, architecture views, clean core environment verification and source-release checks passed. Clean model-less core regression: 1,837 passed, 12 skipped. Remote CI is tracked by the checks below.
The evaluation records 100 live Flash requests across successive corpus iterations, not 100 persona passes. Known limitations include irrelevant retrieval for a Chinese greeting, occasional masculine voice, unprompted Christina reactions and invented shared history. Some failures occur without any retrieved reference. These are disclosed; this PR does not claim to solve default-off persona consistency or justify enabling RAG by default.
Related to #56; keep the issue open for contributor retesting and separate persona-baseline follow-up. Setup: optional character knowledge.
Main Chat 对红莉栖 BBS 网名及中日文写法的识别存在缺口。这次接入各聊天路径共用的可选本地检索,并提供有来源的中日文基础资料;默认关闭,固定 persona 不变。分支基于包含 #55 的公开主线
d5e1d7f。复用 FAISS/E5/JSON 基础方式,支持知识目录和文件,附 15 个主题、30 条资料。开关、索引目录、top-k 和阈值可通过
.env或 Settings 配置;显示实际生效值、加载状态和最近检索摘要,CLI 可查看被过滤的候选。资料作为明确标注的本轮参考传入,保留原始用户输入和历史;命中文字会发给所选模型,包括远程 API。模型仅在显式准备时下载,聊天运行时使用本地缓存。新增可选
rag依赖和 CPU 检索 CI,保持现有 CPU/cu124/ROCm 构建选择,补齐中英文 README、配置及评估说明。未上传旧个人库、索引或模型。验证:107 项相关 Python 测试通过、1 项跳过;32 条真实检索开发用例通过;15 项 Electron 测试、前端构建、Ruff、架构视图、干净 core 环境及源码发布检查通过。干净且未安装模型依赖的 core 完整回归:1,837 项通过、12 项跳过。远程 CI 状态见下方检查。
评估记录了多个资料迭代中的 100 次真实 Flash 调用,不是 100 次角色测试全通过。中文闲聊误召回、偶发男性口吻、无端反应“克里斯蒂娜”和虚构共同经历等仍有反例,部分出现在没有检索资料时。这次先完成 RAG 和资料,人设基线另行处理;#56 保持开启,邀请贡献者重建索引并复测。