fix(ci): resilient apt install with per-attempt timeout + retry, drop dead-weight packages

[Liam0205]

Summary

The 2026-07-10 nightly diff-fuzz run (#164) died in "Install C++ build deps": the Azure archive mirror served cmake (11.2 MB) at ~26 KB/s — 433s for that one package — and the plain timeout 600 tripped mid-download (exit 124) before any real work started. Same failure class as #125 (2026-06-18), which was "fixed" by bumping the timeout 300s→600s. A bigger static timeout just moves the cliff.

Root cause

1. Single-shot install with a whole-phase timeout: one crawling mirror rotation burns the entire budget; there is no second chance even though rotation usually lands a healthy endpoint on retry.
2. Dead-weight packages widen the exposure window: the fatal download in Nightly diff-fuzz: workflow failed (2026-07-10) #164 was cmake — which the runner image already preinstalls (3.31+, newer than apt's 3.28, and earlier in PATH; the apt copy was never used). ninja-build has no -G Ninja reference anywhere in the repo; build-essential/g++-13 are preinstalled on noble; libhiredis-dev is unused (pine-cpp redis client is raw-socket). 15.7 MB of archives on the Nightly diff-fuzz: workflow failed (2026-07-10) #164 manifest vs ~1.5 MB actually needed — a ~10x larger slow-mirror surface.

Changes

• scripts/ci-apt-install.sh (new): shared resilient wrapper — apt-get update + install, up to 3 attempts each under a per-attempt timeout (default 300s), backoff between attempts, dpkg --configure -a repair after a mid-unpack kill, Acquire::Retries=3 (in-attempt connection drops) + DPkg::Lock::Timeout=60 (competing lock holders).
• All 12 apt sites migrated across ci.yml (7) / nightly-diff-fuzz / daily-sanitized-fuzz / nightly-benchmark / nightly-sanitizer (2), with package lists slimmed to what the image actually lacks (libluajit-5.1-dev, libcurl4-openssl-dev, plus util-linux for sanitizer jobs and redis-server=5:7.* for cross-validate).
• Install steps now assert preinstalled tools (cmake --version / g++-13 --version) so a future runner-image change fails loudly at install time instead of at first compile.

Worst case per site: 3 × 300s + backoff ≈ 15.5m (vs old 20m budget); expected path is a single ~30s attempt.

Test plan

• bash -n + workflow YAML parse (all 8 files)
• Mocked-sudo scenarios: hang-then-recover (attempt 1 killed at per-attempt timeout, attempt 2 succeeds, exit 0), hard failure (exit 1 after N attempts), no-args (usage, exit 2), versioned-glob passthrough (redis-server=5:7.* reaches apt verbatim, unexpanded)
• All migrated steps run at repo root (no working-directory interference), checkout precedes every call
• This PR's own CI runs the migrated ci.yml sites live — cpp-build / cpp-sanitizer / cpp-tsan / cpp-lint / cpp-test / cross-validate / differential-fuzz all exercise the new script

Closes #164.

[github-actions]

🔍 PR 审查

项目	结果
结论	✅ APPROVE
审查截止	c04d73e31e4953a179a765a2a9c38a4d0a154b59

改动干净、动机清晰:用「per-attempt timeout + retry」替代静态 timeout 600,并把镜像慢速暴露面从 ~15.7 MB 收窄到 ~1.5 MB。脚本逻辑正确,workflow 迁移完整。已独立核验 PR 的关键论断。

核验结果:

• ✅ 12 处 bash scripts/ci-apt-install.sh 调用,workflow 里已无残留裸 apt-get(全部迁移)
• ✅ pine-cpp 全树无 hiredis 引用 —— libhiredis-dev 确为死重
• ✅ 全仓库无 -G Ninja / -GNinja —— ninja-build 确为死重
• ✅ 版本 glob redis-server=5:7.* 在 workflow 侧已双引号包裹,经脚本 "$@" 原样透传给 apt,不会被 shell 展开
• ✅ retry() 无 set -e 干扰,依赖显式 || exit 1;kill 后 dpkg --configure -a 修复合理

💬 一点说明 (非阻塞)

• 文件: scripts/ci-apt-install.sh#L44-L46
• 观察: timeout ... sudo apt-get ... 触发超时时,SIGTERM 先发给 sudo,再由 sudo 转发给子进程 apt-get。现代 sudo 会转发信号,且这与改动前的 timeout 600 sudo apt-get 行为一致 —— 不是回归,仅作记录。配合 DPkg::Lock::Timeout=60,即使上一次 apt 残留持锁,下次尝试也能等出来,链路是自洽的。

代码良好,可以合并。

[github-actions]

🔍 PR 增量审查

项目	结果
结论	✅ APPROVE
审查范围	c04d73e..f667e97

增量改动纯文档（llmdoc/ 记录本次 CI apt 修复），无代码变更。已核对文档论断与实际代码一致，可以合并。

核验结果:

• ✅ 12 个 apt 调用站点（ci.yml 7 + nightly-sanitizer.yml 2 + nightly-benchmark.yml/daily-sanitized-fuzz.yml/nightly-diff-fuzz.yml 各 1），与 reflection 中"全部 12 个 apt 站点迁移"一致
• ✅ scripts/ci-apt-install.sh 默认值 ATTEMPTS=3 / ATTEMPT_TIMEOUT=300、backoff attempt*10s、kill 后 dpkg --configure -a，与 guide/reflection 描述逐条吻合
• ✅ 历史归因 Nightly diff-fuzz: workflow failed (2026-06-18) #125（2026-06-18，300→600s）/ Nightly diff-fuzz: workflow failed (2026-07-10) #164（2026-07-10，cmake 11.2 MB @26 KB/s 用 433s）与脚本头注一致
• ✅ 暴露面数字 15.7 MB vs ~1.5 MB 与脚本注释一致
• ✅ llmdoc/index.md 两处指针（guide 摘要行 + reflection 条目）与新增文件对应，无悬挂链接

三份文档（ci-quality-baseline.md 新增 apt 约定小节 + 检索指针、index.md 双指针、新 reflection）内容自洽、与代码同步，符合 llmdoc 文档规范。

审查截止: f667e97