Skip to content

feat(wallpaper): add desktop keyboard chat input - #55

Merged
Lucas1479 merged 4 commits into
Code-Amadeus:mainfrom
xijiangkiki:feat/wallpaper-keyboard-input
Sep 6, 2026
Merged

feat(wallpaper): add desktop keyboard chat input#55
Lucas1479 merged 4 commits into
Code-Amadeus:mainfrom
xijiangkiki:feat/wallpaper-keyboard-input

Conversation

@xijiangkiki

@xijiangkiki xijiangkiki commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

What and why

Wallpaper mode may be used without voice interaction, while reopening the main Chat window can obscure the character and disrupt the wallpaper presentation. This adds a compact keyboard message input panel to Wallpaper mode, allowing users to continue chatting without returning to the main Chat window.

The panel opens by default in wallpaper mode and supports Enter to send, Shift + Enter for a new line, Esc to clear, a Send button, and a close/reopen toggle.

Linked issue: none

Change class

  • Wallpaper UI feature

Owning layer: Wallpaper Electron Slice.

User-visible effect: Users can type and send messages directly from Wallpaper mode.

Compatibility or migration impact: none.

Evidence

Passed:

  • Wallpaper input and session regression tests
  • Local bridge security tests
  • Electron npm test
  • Electron npm run build
  • git diff --check

Manual checks:

  • Toggle works after switching focus to another application.
  • Enter / Shift + Enter / Esc / Send button work as expected.
  • Messages appear once in Main Chat.
  • Existing conversations continue; an empty conversation list creates a new Chat.

Screenshots

Before

image

After

Expanded
image

Collapsed
image

Final check

  • One coherent Wallpaper interaction improvement.
  • No secrets, local sessions, models, voice material, or restricted assets included.

@Lucas1479 Lucas1479 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xijiangkiki — I support adding keyboard input to Wallpaper mode, and reusing the existing Chat/Session entry points is a good direction. CI is green, and I also ran the three related Python test files successfully (26 tests).

I reproduced two issues on e93218d that should be addressed before merging:

  1. Confirmed speculative voice turns no longer produce a visible user message.
    The new chat.user event is emitted only when a turn starts with pending=False. confirm_pending_turn() does not emit it when a speculative turn is confirmed, while the previous wake-ASR message insertion has been removed from ChatPage. With send_text(..., source="wake", pending=True) followed by successful confirmation and stream completion, I observed zero chat.user events.
    Please cover both admission paths through the appropriate owner: an ordinary confirmed turn should emit once; a pending turn should emit once on successful confirmation; repeated confirmation must not duplicate it, and discarded turns must not display a user message.

  2. Enter during IME composition submits the message.
    The composer keydown handler does not distinguish IME composition from a normal Enter press. In an event-level reproduction using the actual composer handler, an Enter event with isComposing=true and keyCode=229 still prevented the default action and triggered a send request. This can interfere with Chinese/Japanese candidate selection.
    Please add composition handling and executable keyboard-event tests: candidate confirmation must not send or clear the draft; ordinary Enter must send, and Shift+Enter must still insert a newline. The current source-string assertions and node --check do not exercise these behaviors.

Please make these changes in this PR and add the regression tests. The feature direction is supported; after these two issues are fixed and the checks pass, I can re-review it for merging.


感谢 @xijiangkiki,我认可 Wallpaper 模式增加键盘输入的方向,复用现有 Chat/Session 入口也很合适。CI 已通过,我另外运行了相关的三个 Python 测试文件,26 项全部通过。

不过,在 e93218d 上额外复现了两处需要合并前修复的问题:

  1. 投机语音确认后,主聊天会漏掉用户消息。
    新的 chat.user 只在 pending=False 的发送入口发出,confirm_pending_turn() 确认投机轮次时没有补发,而 ChatPage 原来的 wake-ASR 消息追加逻辑已经删除。实际调用 send_text(..., source="wake", pending=True),再成功确认并等待流结束,收到的 chat.user 事件仍为 0
    请在相应的生命周期所有者处覆盖两条路径:普通确认轮次显示一次;pending 轮次确认成功后显示一次;重复确认不重复显示;作废轮次不显示用户消息,并补回归测试。

  2. 中文/日文输入法选词时按 Enter 会误发送。
    新输入框没有区分组合输入状态。在调用真实 composer 事件处理函数的复现中,isComposing=truekeyCode=229 的 Enter 事件仍然阻止了默认行为并发出发送请求。
    请补输入法组合输入处理和可执行的键盘事件测试:确认候选词不能发送或清空草稿;普通 Enter 仍能发送,Shift+Enter 仍能换行。当前的源码字符串断言和语法检查没有覆盖这些行为。

建议就在这条 PR 中完成这两处修复和回归测试。功能方向是认可的,修复并通过检查后,我再复核合并。

@xijiangkiki

Copy link
Copy Markdown
Contributor Author

Addressed both review findings.

  1. Pending speculative turns now emit chat.user exactly once after successful confirmation; discarded and repeated confirmations do not display duplicate user messages.
  2. The Wallpaper keyboard composer now leaves IME composition events to the input method, with executable keyboard-event coverage for IME Enter, ordinary Enter, and Shift+Enter.

Validation:

  • npm test
  • npm run build
  • tests/test_pending_turn.py
  • tests/test_wallpaper_asset_revision.py
  • tests/test_local_bridge_security.py

@Lucas1479 Lucas1479 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @xijiangkiki for addressing both findings and adding executable regression coverage. I checked the latest fixes against the current mainline integration: all 28 related Python tests and the three keyboard-event tests passed, and all five CI checks are green. The speculative user-message lifecycle and IME candidate confirmation now address the issues from the previous review. Approved for merging.


感谢 @xijiangkiki 完成两处修复,并补上可执行的回归测试。我已按当前主线的合并版本复核:28 项相关 Python 测试和 3 项键盘事件测试全部通过,五项 CI 也全绿。投机用户消息的确认/作废行为,以及输入法候选词确认,均已解决上一轮指出的问题。本次审核通过,准备合并。

@Lucas1479
Lucas1479 merged commit d5e1d7f into Code-Amadeus:main Sep 6, 2026
5 checks passed
@xijiangkiki
xijiangkiki deleted the feat/wallpaper-keyboard-input branch September 6, 2026 09:02
Lucas1479 added a commit that referenced this pull request Sep 6, 2026
…ge (#59)

Main Chat can miss Kurisu's canonical BBS handle, especially across
Chinese/Japanese spellings. This adds optional local character retrieval
shared by all eight Chat routes, with a sourced bilingual starter
corpus. RAG is **off by default** and the static persona is unchanged.
Based on current public main `d5e1d7f`, including #55.

- Reuse FAISS + multilingual E5 and JSON source data. Build from a file
or directory; include 15 topics / 30 Chinese and Japanese records.
Personal data, index binaries and weights are not published.
- Configure enablement, index directory, top-k and threshold through
`.env` or Settings. Show applied configuration, loading/failure state
and last retrieval summary. CLI diagnostics show rejected candidates as
well as accepted hits.
- Label retrieved passages as fallible current-turn reference data,
preserve the original user message/history and existing fallback
behavior, and reuse the cache-only model-loading boundary. Accepted
excerpts go to the selected model, including remote APIs.
- Add the optional `rag` extra without changing CPU/cu124/ROCm build
selection; add a dedicated CPU retrieval CI job. Update both READMEs,
setup/evaluation docs and source-archive selection.

Validation: 107 targeted Python tests passed (1 skipped); all 32 real E5
retrieval development cases passed; 15 Electron tests and
TypeScript/Vite build passed. Ruff, architecture views, clean core
environment verification and source-release checks passed. Clean
model-less core regression: 1,837 passed, 12 skipped. Remote CI is
tracked by the checks below.

The
[evaluation](https://github.com/Code-Amadeus/Amadeus/blob/codex/optional-character-rag/docs/character_rag_evaluation.md)
records **100 live Flash requests across successive corpus iterations**,
not 100 persona passes. Known limitations include irrelevant retrieval
for a Chinese greeting, occasional masculine voice, unprompted Christina
reactions and invented shared history. Some failures occur without any
retrieved reference. These are disclosed; this PR does not claim to
solve default-off persona consistency or justify enabling RAG by
default.

Related to #56; keep the issue open for contributor retesting and
separate persona-baseline follow-up. Setup: [optional character
knowledge](https://github.com/Code-Amadeus/Amadeus/blob/codex/optional-character-rag/docs/character_rag.md).

---

Main Chat 对红莉栖 BBS
网名及中日文写法的识别存在缺口。这次接入各聊天路径共用的可选本地检索,并提供有来源的中日文基础资料;**默认关闭,固定 persona
不变**。分支基于包含 #55 的公开主线 `d5e1d7f`。

复用 FAISS/E5/JSON 基础方式,支持知识目录和文件,附 15 个主题、30 条资料。开关、索引目录、top-k 和阈值可通过
`.env` 或 Settings 配置;显示实际生效值、加载状态和最近检索摘要,CLI
可查看被过滤的候选。资料作为明确标注的本轮参考传入,保留原始用户输入和历史;命中文字会发给所选模型,包括远程
API。模型仅在显式准备时下载,聊天运行时使用本地缓存。

新增可选 `rag` 依赖和 CPU 检索 CI,保持现有 CPU/cu124/ROCm 构建选择,补齐中英文
README、配置及评估说明。未上传旧个人库、索引或模型。

验证:107 项相关 Python 测试通过、1 项跳过;32 条真实检索开发用例通过;15 项 Electron
测试、前端构建、Ruff、架构视图、干净 core 环境及源码发布检查通过。干净且未安装模型依赖的 core 完整回归:1,837 项通过、12
项跳过。远程 CI 状态见下方检查。

评估记录了多个资料迭代中的 100 次真实 Flash 调用,**不是 100
次角色测试全通过**。中文闲聊误召回、偶发男性口吻、无端反应“克里斯蒂娜”和虚构共同经历等仍有反例,部分出现在没有检索资料时。这次先完成 RAG
和资料,人设基线另行处理;#56 保持开启,邀请贡献者重建索引并复测。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants