Skip to content

Commit 25ece82

Browse files
committed
fix(sqlite): scan json text columns safely
1 parent 22dc857 commit 25ece82

7 files changed

Lines changed: 274 additions & 48 deletions

File tree

.planning/STATE.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ milestone: v4.2.0
44
milestone_name: 容器合并 · SQLite 迁移 · 配置统一
55
status: Awaiting next milestone
66
stopped_at: Phase 58 context gathered
7-
last_updated: "2026-07-08T18:47:40+08:00"
8-
last_activity: 2026-07-08 — Completed quick task 260708-q1b: 降低 VNC 与 Chromium 默认资源占用
7+
last_updated: "2026-07-09T15:02:05+08:00"
8+
last_activity: 2026-07-09 — Completed quick task 260709-kqq: 修复 SQLite JSON TEXT 扫描失败
99
progress:
1010
total_phases: 5
1111
completed_phases: 1
@@ -28,7 +28,7 @@ See: .planning/PROJECT.md (updated 2026-05-14)
2828
Phase: Milestone v4.2.0 complete
2929
Plan: —
3030
Status: Awaiting next milestone
31-
Last activity: 2026-07-08 — Completed quick task 260708-q1b: 降低 VNC 与 Chromium 默认资源占用
31+
Last activity: 2026-07-09 — Completed quick task 260709-kqq: 修复 SQLite JSON TEXT 扫描失败
3232

3333
## Accumulated Context
3434

@@ -81,6 +81,7 @@ v3.6 关键技术决策(已落地,可作为后续里程碑参考):
8181
| 260602 | 规则导入导出按钮 | 2026-06-02 | pending | [260602-rules-import-export](./quick/260602-rules-import-export/) |
8282
| 260707-qw2 | VNC 状态展示、启动重启控制、容器内自动拉起与 Chromium 窗口适配 | 2026-07-07 | pending | [260707-qw2-vnc-chromium](./quick/260707-qw2-vnc-chromium/) |
8383
| 260708-q1b | 降低 VNC 与 Chromium 默认资源占用 | 2026-07-08 | pending | [260708-q1b-vnc-chromium-vnc](./quick/260708-q1b-vnc-chromium-vnc/) |
84+
| 260709-kqq | 修复 SQLite JSON TEXT 扫描失败 | 2026-07-09 | pending | [260709-kqq-bypass-preset-rules-sqlite](./quick/260709-kqq-bypass-preset-rules-sqlite/) |
8485

8586
### Roadmap Evolution
8687

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Quick Task 260709-kqq: 修复 bypass preset rules SQLite 扫描失败
2+
3+
## Goal
4+
5+
修复生产环境 `list bypass presets failed` 反复报错,原因是 SQLite `TEXT` JSON 列不能直接扫描到 `json.RawMessage`
6+
7+
## Tasks
8+
9+
1. 用真实 SQLite migration 写回归测试,复现 `host_bypass_presets.rules` 扫描失败。
10+
2. 梳理同类 JSON TEXT 字段,避免只修 `rules` 单点。
11+
3. 增加通用 JSON 文本扫描 helper,并更新 bypass preset / snapshot / audit log 扫描路径。
12+
4. 跑 repository 相关测试和格式检查。
13+
5. 说明这类错误与 managed CPU、VNC/SSH 不可连接之间的可能关系与部署动作。
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
status: complete
3+
phase: quick-260709-kqq
4+
subsystem: sqlite, repository, bypass, runtime
5+
tags: [sqlite, json, bypass, egress, host-lifecycle]
6+
---
7+
8+
# Quick Task 260709-kqq: 修复 bypass preset rules SQLite 扫描失败 Summary
9+
10+
## 结果
11+
12+
- 修复 SQLite `TEXT` JSON 列直接扫描到 `json.RawMessage` 失败的问题。
13+
- 新增 `jsonText` 扫描 helper,支持 `string``[]byte``NULL` 三种 SQLite 返回形态。
14+
- `host_bypass_presets.rules`、bypass snapshot JSON、bypass audit log JSON 改为先扫描到 `jsonText`
15+
- `hosts.host_mounts``egress_ips.proxy_config` 的读取路径也改为 `jsonText`,避免主机详情、VNC/SSH 控制、`prepare_host` 读取出口配置时踩同类问题。
16+
- 新增真实 SQLite migration 回归测试,覆盖 bypass preset、host mounts、egress proxy config 与 `GetEgressIPByHost`
17+
18+
## 根因
19+
20+
`database/sql` 不能把驱动返回的 `string` 直接写入 `*json.RawMessage`,线上错误:
21+
22+
`unsupported Scan, storing driver.Value type string into type *json.RawMessage`
23+
24+
对应字段是 SQLite `TEXT` JSON 列,不是 Go 原生 JSON 类型。
25+
26+
## 验证
27+
28+
- `go test ./internal/store/repository -run 'Test(BypassRepository_ListPresetsScansSQLiteTextRules|Repository_ScansSQLiteTextJSONForHostAndEgress)' -count=1` 通过。
29+
- `go test ./internal/store/repository -count=1` 通过。
30+
- `go test ./internal/runtime/tasks -count=1` 通过。
31+
- `go test ./internal/controlplane/http -count=1` 通过。
32+
- `go test ./... -count=1` 通过。
33+
34+
## 说明
35+
36+
这会停止 `/v1/admin/bypass/presets` 反复 500 与日志刷屏。由于 `prepare_host` 会读取 `egress_ips.proxy_config`,同类扫描失败也可能导致主机准备失败,进而表现为 VNC 和 SSH 都连不上。修复部署后,已有失败任务需要重新触发主机 prepare/start/rebuild。
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
package repository
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
)
7+
8+
// jsonText scans SQLite TEXT/NULL JSON columns into bytes before callers decode
9+
// them into typed fields. database/sql cannot scan a driver string directly into
10+
// json.RawMessage because RawMessage is a named []byte type.
11+
type jsonText []byte
12+
13+
func (j *jsonText) Scan(value any) error {
14+
switch v := value.(type) {
15+
case nil:
16+
*j = nil
17+
return nil
18+
case string:
19+
*j = append((*j)[:0], v...)
20+
return nil
21+
case []byte:
22+
*j = append((*j)[:0], v...)
23+
return nil
24+
default:
25+
return fmt.Errorf("unsupported JSON text source type %T", value)
26+
}
27+
}
28+
29+
func (j jsonText) RawMessage() json.RawMessage {
30+
if len(j) == 0 {
31+
return nil
32+
}
33+
out := make([]byte, len(j))
34+
copy(out, j)
35+
return json.RawMessage(out)
36+
}

0 commit comments

Comments
 (0)