Commit 8e26d14
refactor(plugin-sdk): push_message v2 — visibility/ai_behavior 双轴 + parts 列表 (#1046)
* refactor(plugin-sdk): push_message v2 — visibility/ai_behavior 双轴 + parts 列表
把 push_message 从 message_type+delivery+content 单轴枚举重写成两条正交轴
(visibility / ai_behavior) + OpenAI 风格 parts 列表,让"用户看到哪儿"和
"AI 怎么处理"分开表达,并支持 text+media 同帧推送(game agent 截图流场景)。
新签名(plugin/sdk/shared/core/push_message_schema.py 单一权威):
ctx.push_message(
visibility=[], # ["chat"] / ["hud"] / [] / 组合
ai_behavior="respond", # respond / read / blind
parts=[
{"type":"text","text":...},
{"type":"image","data":bytes,"mime":...},
{"type":"image","url":...},
{"type":"audio","data":bytes,"mime":...},
{"type":"video","url":...},
{"type":"ui_action","action":"media_play_url",...},
{"type":"ui_action","action":"media_allowlist_add","domains":[...]},
],
source=..., target_lanlan=..., metadata=..., priority=...,
)
兼容窗口(v0.9 移除)
message_type / delivery / reply / content / binary_data / binary_url /
mime / description / unsafe 全部仍可用,每用一个 emit DeprecationWarning。
translate_push_message 在 SDK 层做 v1→v2 翻译,wire payload 同时填新旧
字段,下游 query_service 等未迁移的 reader 透明继续工作。
直接删除(无 compat)
SdkContext.register_music_domains() —— 全仓 grep 没有 in-tree 调用。
下游链路
proactive_bridge 重写:读 v2 字段,按 parts 分流到既有 legacy event_types
(proactive_message / music_play_url / music_allowlist_add),media 挂在
proactive_message.media_parts,main_server 分发时 base64 解码后调
session.send_media_input 注入到 realtime session。
迁移
bilibili_danmaku / memo_reminder / sts2_autoplay 三个 in-tree 插件已迁到新
签名,零 warning 触发;PLUGIN_DEVELOPMENT_GUIDE.md 全节重写;
docs/changelog/plugin-push-message-v2.md 详述动机 / 对照表 / 移除清单。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(push-message-v2): address PR review — stream_image API, compat fields, target_lanlan
修了首轮 review 提到的 7 个真实问题(一处 reviewer 误判已在 thread 反驳):
- main_server.py: 媒体注入用错了 API。realtime client 只暴露 stream_image
(image_b64) 和 stream_audio (PCM bytes),不存在 send_media_input。改成
调 stream_image 并直接传 base64 字符串;audio/video 现阶段无对应通道,
log warning 后丢弃。URL-only 媒体也改成 warn-and-drop(v0.9 引入 fetch
worker 时再补上)。
- _types/protocols.py: push_message Protocol 改成 keyword-only,跟
sdk/shared/core/{context,types}.py 的实现一致,避免 type checker
通过 + 运行时 TypeError 的分裂。
- core/context.py 兼容层:现在也把 binary_data (bytes)、mime、delivery、
reply 落到 wire payload 顶层,确保 query_service 等还没迁移到 v2 的
reader 在 deprecation window 里能继续读到完整 v1 形状。binary_data 从
parts 的 binary_base64 反解出来(仅取首个 image/audio/video part)。
- proactive_bridge.py: 删 SCHEMA_VERSION 未用 import;result_parser fallback
的 except 加 logger.debug 不再静默吞错;music_play_url /
music_allowlist_add 事件多带 lanlan_name=target_lanlan 字段,让 plugin
显式 target_lanlan 时 main_server 能路由到指定 session(之前会退化广播)。
- bilibili_danmaku: _push_to_ai 把 self._target_lanlan 透传给 push_message
(set_target_lanlan 入口配置的 AI 终于真正生效)。
- PLUGIN_DEVELOPMENT_GUIDE.md: 把失效的 #push_message_kwargs---object 链接
指向新加的 <a id="push-message-v2"></a> 锚点。
未采纳
- _synthesize_legacy_message_type 的 visibility 校验:(visibility=[],
ai_behavior=respond) 是 v2 最常用的 "让 AI 转述" 模式,对应的 legacy
discriminator 就是 proactive_notification。加 visibility 检查会让这个
最常见的 case 落到 "text",反而是退化。已在 thread 反驳。
- bilibili 的 target_lanlan 据说"重构丢了"——其实重构前这个字段也没在传,
是 pre-existing missing;不过修起来零成本,顺手补上了。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(plugin-guide): 标注 v2 push_message media parts 的当前实现限制
CodeRabbit 指出文档把 image/audio/video 写得像三种都能直接喂给 AI,但
main_server 这一版只接 inline image (走 stream_image),URL 形态和
audio/video 都会 warn-drop。补一段「当前实现限制」明确说明,并把上面
example 里的 image-url / audio / video 三行改成注释占位,避免插件作者
照抄后发现 AI 实际看不到。
Refs: PR #1046 review thread r3168...
* docs(plugin-guide): 迁移表显式标 ai_behavior="blind"
CodeRabbit nit:music_* 两条迁移行只写 parts + visibility,读者
可能以为保留默认 respond 也行。补 ai_behavior="blind" 跟例 4/6
和 proactive_bridge fan-out 的语义对齐。
Refs: PR #1046 review thread
* lint(docs): forbid relative-up markdown links inside docs/
docs/ 走 VitePress,site root 就是 docs/ 自己。任何 markdown 链接以
\`..\` 开头都会在 deploy 时解析到 docs 外面、构建直接爆炸——这次
changelog 的 \`../../plugin/sdk/...\` 是第二次(?)触发同一个坑。
加 lint 把这条钉死,下次写错文档时 CI 直接拦下,不再让它跑到 main
触发 docs.yml 的 deploy 失败。
新加的 lint
- scripts/check_docs_no_relative_paths.py 扫 docs/**/*.md,匹配
\`](...\` 形态的 markdown 链接;任何 target 以 \`..\` 开头都报错
- 不抓 fenced code block 里的 \`cd ../..\` 这类 shell 字面量
(那不是 markdown 链接,对 VitePress 无害)
- 没有 per-line 跳过——加了就把这条 lint 的存在意义瓦解了
- analyze.yml 新增一步运行该脚本,job 名字也加 docs-no-relative-paths
修了的 5 份文档(共 52 处链接)
- docs/changelog/plugin-push-message-v2.md:上一轮我自己加的 3 处
(../../plugin/...) 改成 inline code
- docs/design/home-tutorial-yui-guide-{main-owner,performance-owner,
preparation-freeze,three-person-collaboration}-stage-breakdown.md:
pre-existing 的 ../../static/*.js 共 49 处,统一去掉链接外壳改成
\`static/foo.js\` 形态——同时移除 changelog 文档里残留的过期描述
send_media_input → stream_image。
* fix(docs-lint): 跳过 fenced code block;细化 changelog video 描述
CodeRabbit 两条 review:
1. scripts/check_docs_no_relative_paths.py 没跳 ```/~~~ 围起来的 fenced
code block,文档里写"反面示例"会被自己拦下。加个简单的 fence
状态机,遇到 ``` / ~~~ 切 in_fence,跳过其中所有 link 匹配。
测过:fenced 内 [bad](../foo.md) 不再报,fenced 外的照旧报。
2. changelog 迁移表写 "(or audio/video)" 容易让人以为 video.data 也
被 AI 注入消费。其实 schema 接受任意 image/audio/video 的 data
字段,但 main_server 这版只 stream_image 走通了,audio/video 都
warn-drop。明确写成 "or audio; video accepted in schema but
main_server warn-drops it for now" 跟 PLUGIN_DEVELOPMENT_GUIDE 的
"当前实现限制"段对齐。
Refs: PR #1046 review threads
* chore(push-message-v2): TODO(v0.9) 标记 description 字段全部清理点
description 在 v2 schema 里没有任何语义角色——只是 v1 通道里给
log line 和 query_service 当 human label 的兜底。当前留着完全是
为了 v1 兼容窗口。本提交把所有还在 surface description 的位置打
TODO(v0.9) 标记,方便 v0.9 cleanup PR 一次 grep 全删:
- plugin/core/context.py: legacy_description 合成处
- plugin/plugins/{bilibili_danmaku,memo_reminder,sts2_autoplay}:
metadata={"description": ...} 三处(迁移时顺手留了,但其实
metadata.description 也没人读)
- plugin/server/application/messages/query_service.py: 序列化
response 时把 record.get("description") 反弹回去那一处
- docs/changelog/plugin-push-message-v2.md: 在 "Removed in v0.9"
列表里单独点名 description,列出全部 TODO(v0.9) 位置方便定位
行为零变化:description 仍按现状走完整 v1 通道;只是现在 v0.9
cleanup PR 可以 \`grep "TODO(v0.9)"\` 全找出来。
* fix(push-message-v2): legacy description 顶层从 metadata.description 兜底
CodeRabbit 指出:上一 commit 给 3 个迁移插件加 TODO(v0.9) 时,把
description 放进了 metadata["description"],但 _build_wire_payload
里 legacy_description 只看 kwarg、没看 metadata。结果 v2 调用方推的
消息到 query_service / 老日志那一头会提前丢标签——deprecation window
内不该提前失效。
补三档优先级回填:
1. 显式 description= kwarg(v1 调用方)
2. metadata["description"](已迁到 v2、把 label 放 metadata 里的)
3. 空串(纯 v2 native,从不设 label)
行为:v2 迁移过的 plugin 走 #2,wire payload 顶层 description 不再
丢;v0.9 cleanup 时三档一起删(TODO 注释已更新)。
Refs: PR #1046 review thread
---------
Co-authored-by: Hongzhi Wen <cartabio.coder1@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 91f8a78 commit 8e26d14
21 files changed
Lines changed: 1425 additions & 444 deletions
File tree
- .github/workflows
- docs
- changelog
- design
- plugin
- _types
- core
- plugins
- bilibili_danmaku
- memo_reminder
- sts2_autoplay
- sdk
- plugin
- server
- application/messages
- messaging
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
Lines changed: 5 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | | - | |
| 54 | + | |
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
62 | | - | |
| 61 | + | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
141 | | - | |
| 141 | + | |
142 | 142 | | |
143 | 143 | | |
144 | 144 | | |
| |||
Lines changed: 14 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | | - | |
| 30 | + | |
| 31 | + | |
32 | 32 | | |
33 | | - | |
34 | | - | |
35 | | - | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
112 | | - | |
| 112 | + | |
113 | 113 | | |
114 | 114 | | |
115 | 115 | | |
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
129 | | - | |
| 129 | + | |
130 | 130 | | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
135 | 135 | | |
136 | 136 | | |
137 | 137 | | |
| |||
273 | 273 | | |
274 | 274 | | |
275 | 275 | | |
276 | | - | |
| 276 | + | |
277 | 277 | | |
278 | 278 | | |
279 | 279 | | |
| |||
438 | 438 | | |
439 | 439 | | |
440 | 440 | | |
441 | | - | |
| 441 | + | |
442 | 442 | | |
443 | 443 | | |
444 | 444 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
265 | 265 | | |
266 | 266 | | |
267 | 267 | | |
268 | | - | |
| 268 | + | |
269 | 269 | | |
270 | 270 | | |
271 | 271 | | |
272 | 272 | | |
273 | 273 | | |
274 | 274 | | |
275 | | - | |
276 | | - | |
| 275 | + | |
| 276 | + | |
277 | 277 | | |
278 | | - | |
279 | | - | |
| 278 | + | |
| 279 | + | |
280 | 280 | | |
281 | 281 | | |
282 | 282 | | |
| |||
334 | 334 | | |
335 | 335 | | |
336 | 336 | | |
337 | | - | |
| 337 | + | |
338 | 338 | | |
339 | 339 | | |
340 | 340 | | |
| |||
0 commit comments