fix(macos): lazy-load Windows pointer forwarding - #48
Conversation
There was a problem hiding this comment.
审查通过。这个改动把类型提示导入放到 TYPE_CHECKING,并只在现有 sys.platform == "win32" 且确实启动滚轮转发时进行运行时导入,修复位置和所有权都正确,没有改变 Windows 行为。
我在 Windows 上复核了相关套件:15 项通过、1 项非 Windows 回归测试按条件跳过;另外确认普通导入不会提前加载 wallpaper.pointer_wheel_forwarder,Ruff 与 git diff --check 通过。与 #46 的合并分析也没有冲突。
该 PR 可以作为独立的跨平台导入修复;批准本身不代表 macOS 原生 Wallpaper 宿主已经完成。
English
Approved. This change moves the type-only import behind TYPE_CHECKING and imports the Windows pointer forwarder at runtime only when the existing sys.platform == "win32" branch actually starts wheel forwarding. The fix is located at the correct owner and does not change Windows behavior.
I reran the relevant suites on Windows: 15 tests passed and the one non-Windows regression test was skipped as intended. I also confirmed that a normal module import does not eagerly load wallpaper.pointer_wheel_forwarder; Ruff and git diff --check passed, and the merge analysis with #46 was clean.
This PR can land as an independent portability fix. The approval does not mean that the native macOS Wallpaper host is complete.
## What and why The embedded GPT-SoVITS path assumes CUDA in several runtime-only code paths: device selection defaults to CUDA, synchronization and device contexts call `torch.cuda` unconditionally, and BigVGAN probes/loads CUDA kernels even when inference runs on another backend. On Apple Silicon this prevents the bundled local TTS backend from running through PyTorch MPS. This PR adds the smallest runtime compatibility layer needed for Apple Silicon: - resolve `TTS_DEVICE=auto` to `mps` on Apple Silicon and `cpu` on Intel macOS; - validate requested CUDA/MPS devices with actionable errors; - use device-neutral context and synchronization helpers; - keep BigVGAN CUDA kernels on CUDA while using its PyTorch implementation on MPS/CPU; - load supported GPT-SoVITS checkpoints with PyTorch 2.6 safe loading; - enable PyTorch MPS CPU fallback for the Electron-launched backend. Performance-policy changes from the local macOS prototype—lower sample steps, frame-rate caps, lazy animation loading, static secondary displays, and backend auto-restart—are intentionally excluded. Linked Issue for product-semantic or public-contract changes: #47 Related dependency-profile work: #46. This PR intentionally does not add Darwin packages to the CUDA-named `local-cu124` extra; the install-profile contract can be integrated with the ongoing dependency migration. ## Change class - [ ] Routine fix, documentation, test, maintenance, or presentation-only UI - [x] Product-semantic or public-contract change discussed in the linked Issue - [ ] Isolated, default-off experiment Owning layer: embedded GPT-SoVITS runtime and backend process environment User-visible effect, or `none`: Apple Silicon users can select `TTS_DEVICE=auto` and run compatible GPT-SoVITS v3 models through MPS. Compatibility or migration impact, or `none`: Explicit indexed CUDA, `mps`, and `cpu` values remain supported. Apple Silicon now resolves `auto`/unindexed `cuda` to `mps`; Intel macOS resolves them to `cpu`. A compatible PyTorch/MPS environment is still required; dependency installation is intentionally left to #46. ## Evidence Commands and manual journeys run: - New device/checkpoint tests — 11 passed - Existing relevant TTS/config/profile tests — 84 passed - Focused Ruff checks — passed - Electron `npm run build` — passed - Electron existing tests — 4 passed - Apple Silicon verifier: Python 3.12.13, torch/torchaudio 2.6.0, `mps_built=True`, `mps_available=True` - Manual GPT-SoVITS v3 synthesis on an M1 Max using the local model package — passed - `git diff --check` — passed The full macOS collection on current `main` is independently blocked by an eager import of the Windows-only pointer hook; #48 contains the isolated fix. The remaining 23 macOS failures (temporary-path alias expectations and Windows desktop-layer assertions) reproduce unchanged on `upstream/main`. - [x] Relevant Python tests pass - [x] CPU/model-less baseline remains supported - [x] Electron `npm run build` passes when Electron code changed - [ ] Dependency audit passes when dependencies changed - [ ] Before/after screenshots are attached for visible UI changes - [x] Documentation/examples are updated for changed settings or contracts ## Final check - [x] This PR addresses one coherent problem without unrelated cleanup - [x] It does not add a speculative API, fallback, or compatibility path - [x] No secrets, local state, model weights, voice material, or restricted assets are included - [x] Third-party notices and provenance are preserved --------- Co-authored-by: Lucas1479 <sli776@aucklanduni.ac.nz>
) This PR builds on #45 and preserves the original contributor commits. It qualifies one exact uv-managed project environment before changing the maintained installation path. The default capability ladder is L1 core → L2 remote voice → L3 CPU VAD → L4 Windows cu124 local models. Every tier uses the project `.venv`, and CPU, NVIDIA, and ROCm Torch builds are mutually exclusive. CPU VAD and cu124 now use PyTorch/Torchaudio 2.6.0 as the maintained baseline. This is the smallest official cu124 upgrade that fixes critical advisory GHSA-53q9-r3pm-6pq6; the previous 2.5.1 lock was rejected by dependency review. The Windows local-model profile also uses the `pyopenjtalk-plus` CPython 3.12 wheel so a clean install does not depend on compiling pyopenjtalk inside a long checkout path. This branch includes the merged portability, Apple Silicon MPS runtime, native macOS menu, and dependency-review changes from #48, #49, #51, and #52. An opt-in `local-rocm` candidate is also included for Windows. It locks AMD's official ROCm 7.2.1 / Torch 2.9.1 packages as a third build selection in the same `.venv`. Qwen ASR and GPT-SoVITS may run in persistent sidecar processes while using that interpreter; sidecar mode is disabled unless explicitly selected. NVIDIA CUDA Graph, NVIDIA BigVGAN kernels, and unrelated performance tuning are not enabled by this candidate. The latest integration includes public main `f7e57e2` and fixes the offline-loading boundary: inherited online flags, already-imported Hub/Transformers state, and cached/custom Hub sessions are now reset before local voice-model loading. Explicit local-file loading is applied to Qwen ASR, BERT, and BigVGAN. The new regression checks block remote requests while still loading a tiny locally generated BERT model. They ran successfully with the qualified cu124 and ROCm libraries. Current head `f1f6397` has all six remote checks green; the current Windows model-less suite reports 1769 passed / 11 skipped, and the Electron model-less smoke passed. Recorded validation (previous L4 qualification and latest integration checks): - `uv.lock` resolves CPU 2.6.0, cu124 2.6.0, and ROCm 2.9.1/7.2.1 branches; invalid combinations fail closed. - A clean `local-cu124` sync passed the 234-package environment contract and `uv pip check`. - On an RTX 4070 Ti SUPER, PyTorch 2.6.0+cu124 completed real CUDA matrix compute, safely loaded the existing GPT and SoVITS v3 checkpoints, and generated a finite 1.612-second / 24 kHz v3 TTS sample through BERT, CNHubert, GPT, SoVITS LoRA, and BigVGAN. - The full L4 Python suite passed: 1783 passed, 2 skipped. - Exact sync back to L1+dev removed 127 voice/model packages and verified that Torch, Qwen ASR, ONNX Runtime, and pyopenjtalk were absent. - A clean `local-rocm` sync installed the fixed candidate and passed its version/import contract plus `uv pip check`. - Earlier revisions passed the remote Windows model-less, single-venv ladder, macOS voice, ROCm clean-install, and Electron build jobs. The Torch 2.5.1 critical advisory is removed. Dependency review has narrowly documented four temporary exceptions: NLTK GHSA-8mgp-746c-j5xp has no patched release, while qwen-asr 0.0.6 requires Transformers 4.57.6 exactly and therefore cannot consume the fixes for GHSA-29pf-2h5f-8g72, GHSA-fgcw-684q-jj6r, and GHSA-xrqw-3rrv-vx5w. Amadeus does not call the affected NLTK persistence, Transformers LightGlue, or `save_pretrained` paths; ASR resolves a local directory and both ASR and TTS force Transformers/Hugging Face offline. Each exception must be removed when a compatible fix is published. All six remote checks have now passed again for `f1f6397`. - Ruff, workflow YAML validation, lock consistency, focused profile/sidecar tests, and diff checks passed. ROCm evidence remains explicitly experimental. Community history records successful RX 9070 XT ASR/TTS sidecars on another ROCm/PyTorch build. On the maintainer's Radeon 780M, the fixed 7.2.1 build installed and enumerated gfx1103, but its first FP32 tensor operation crashed in `amdhip64_7.dll`; Radeon 780M is absent from AMD's Windows support matrix and is not treated as a supported result. A supported AMD GPU still needs to complete the fixed-combination ASR/TTS, microphone/playback, interruption, lifecycle, and long-running journeys. The PR is ready for maintainer review. ROCm remains experimental until supported AMD hardware completes the remaining real-device acceptance. No model weights, recordings, transcripts, generated audio, credentials, virtual environments, or validation caches are committed. Refs #44 and #45. ### 中文摘要 本 PR 基于 #45,并保留原贡献者提交,用于在切换维护基线前验证由 uv 管理的单一项目环境。 默认能力阶梯是 L1 core → L2 远程语音 → L3 CPU VAD → L4 Windows cu124 本地模型。所有梯级共用项目 `.venv`,CPU、NVIDIA 和 ROCm Torch 构建两两互斥。CPU VAD 与 cu124 现以 PyTorch/Torchaudio 2.6.0 作为正式维护基线;这是仍提供官方 cu124 wheel、同时修复 critical 漏洞 GHSA-53q9-r3pm-6pq6 的最小升级。旧 2.5.1 锁正是 dependency review 失败的原因。Windows 本地模型档也统一采用带 CPython 3.12 wheel 的 `pyopenjtalk-plus`,避免在较长仓库路径中现场编译 pyopenjtalk。 本分支现已合入 #48、#49、#51、#52 的跨平台导入、Apple Silicon MPS runtime、macOS 原生菜单和 dependency review 改动。 Windows `local-rocm` 仍是默认关闭的实验候选。它在同一个 `.venv` 中锁定 AMD 官方 ROCm 7.2.1 / Torch 2.9.1,并与 CPU/cu124 构建互斥。Qwen ASR 与 GPT-SoVITS 可使用同一解释器运行在常驻 sidecar 子进程;sidecar 只表示进程隔离,不额外要求虚拟环境,也不会默认启用 NVIDIA CUDA Graph、NVIDIA BigVGAN kernel 或社区补丁中的其他性能调优。 本轮已接上公开主线 `f7e57e2`,并补齐模型离线加载边界:继承的在线环境变量、已导入的 Hub/Transformers 状态和缓存/自定义 Hub HTTP 会话都会在本地语音模型加载前恢复为离线;Qwen ASR、BERT、BigVGAN 的加载也明确只使用本地文件。新增测试验证远程请求被阻止,同时本地生成的小型 BERT 模型仍能正常加载,已在 cu124 与 ROCm 的实际依赖环境中通过。最新提交 `f1f6397` 的六项远程检查全绿,其中 Windows 无模型完整回归为 1769 passed / 11 skipped,Electron 无模型冒烟测试通过。 验证记录(此前 L4 资格验证与本轮集成检查): - `uv.lock` 可解析 CPU 2.6.0、cu124 2.6.0 和 ROCm 2.9.1/7.2.1,冲突组合会明确失败; - 全新 `local-cu124` 同步通过 234 包环境合同和 `uv pip check`; - RTX 4070 Ti SUPER 上,2.6.0+cu124 完成真实 CUDA 矩阵计算,安全加载现有 GPT/SoVITS v3 权重,并经 BERT、CNHubert、GPT、SoVITS LoRA、BigVGAN 生成 1.612 秒、24 kHz 的有限值音频; - 完整 L4 Python 回归 1783 通过、2 跳过; - 同一 `.venv` 精确返回 L1+dev 时移除 127 个语音/模型包,并确认 Torch、Qwen ASR、ONNX Runtime 与 pyopenjtalk 均不存在; - 全新 `local-rocm` 同步通过固定候选版本/导入合同和 `uv pip check`; - 此前提交的远程 Windows 无模型、单 `.venv` 阶梯、macOS voice、ROCm clean-install 与 Electron build 均通过;Torch 2.5.1 critical 漏洞已移除。dependency review 现精确记录四条临时豁免:NLTK GHSA-8mgp-746c-j5xp 尚无修复版;qwen-asr 0.0.6 又严格要求 Transformers 4.57.6,暂时无法采用 GHSA-29pf-2h5f-8g72、GHSA-fgcw-684q-jj6r、GHSA-xrqw-3rrv-vx5w 的 5.x 修复。Amadeus 不调用相关 NLTK 持久化、LightGlue 或 `save_pretrained` 路径;ASR 只解析本地目录,ASR/TTS 均强制 Transformers/Hugging Face 离线。兼容修复发布后必须逐条移除。最新提交 `f1f6397` 的六项远程检查已全部通过; - Ruff、工作流 YAML、锁一致性、profile/sidecar 聚焦测试和 diff check 均通过。 ROCm 继续明确标记为实验候选。社区资料记录了 RX 9070 XT sidecar ASR/TTS 历史成功;维护机 Radeon 780M 上,固定 7.2.1 环境可安装并枚举 gfx1103,但首次 FP32 计算在 `amdhip64_7.dll` 中崩溃。780M 不在 AMD Windows 支持矩阵内,这个负结果只限定本机硬件,不否定社区候选。固定组合仍需由受支持 AMD GPU 完成真实 ASR/TTS、麦克风/播放、打断、生命周期和长时间运行验收。 本 PR 已进入维护者审阅;ROCm 在受支持 AMD 硬件完成剩余实机验收前继续保持实验候选。仓库未提交模型、录音、转写、生成音频、凭证、虚拟环境或验证缓存。关联 #44、#45。 --------- Co-authored-by: Morgan Woods <weiyiding0@gmail.com>
## What and why The existing desktop integration relies on Windows wallpaper hosts and `BrowserWindow.setShape()`. macOS has neither the Lively/Wallpaper Engine host path nor Electron window shaping, so the current Electron slice cannot provide the full wallpaper scene and may be placed below the visible desktop surfaces. This PR adds a native Electron host policy for macOS: - host the existing full wallpaper scene in a desktop-type, click-through window; - place it between the Dock wallpaper surface and Finder's desktop-icon surface; - keep the existing Windows shaped-slice behavior unchanged; - host Canvas in a separate transparent window above desktop icons; - capture the mouse only over renderer-reported Canvas controls; - avoid calling the unsupported macOS `setShape()` API; - expose the existing Canvas presentation calls and fix the Pixi WebGL monitor reference used by the full scene. Frame-rate caps, lazy SpriteForge loading, static secondary-display backgrounds, backend restart policy, chat/ASR UI changes, and application-menu changes from the local prototype are intentionally excluded. Linked Issue for product-semantic or public-contract changes: #47 This branch expects the platform-neutral bridge import fix in #48. Runtime validation was performed with that fix applied locally; it is not duplicated here. ## Change class - [ ] Routine fix, documentation, test, maintenance, or presentation-only UI - [x] Product-semantic or public-contract change discussed in the linked Issue - [ ] Isolated, default-off experiment Owning layer: Electron wallpaper window hosting and the existing web wallpaper scene User-visible effect, or `none`: On macOS, **Wallpaper** opens the full scene on the desktop and keeps Finder icons clickable. Canvas appears in a separate transparent layer and accepts input only over its visible controls. Compatibility or migration impact, or `none`: Windows retains the current shaped interactive slice and helper process. macOS remains a source-run path; signing, notarization, packaging, and multi-display policy are out of scope. ## Evidence Commands and manual journeys run: - `npm test` — 9 passed - `npm run build` — passed - `node --check render/web/wallpaper_scene.js` — passed - `git diff --check` — passed - Authenticated `wallpaper.start` smoke test — `started`, asset port 17778, bridge port 17797 - Canvas renderer committed interactive hit regions (1 initial, then 3 after state load) - Core Graphics window inspection on Apple Silicon: - Dock wallpaper layer: `-2147483624` - Amadeus scene layer: `-2147483609`, on-screen - Finder desktop-icon layer: `-2147483603` - Amadeus Canvas layer: `-2147483598`, on-screen - Runtime log contains no undefined Pixi `app` error and no missing `setCanvasPresentation` / `setAttention` bridge methods after the fix - [ ] Relevant Python tests pass - [x] CPU/model-less baseline remains supported - [x] Electron `npm run build` passes when Electron code changed - [ ] Dependency audit passes when dependencies changed - [ ] Before/after screenshots are attached for visible UI changes - [x] Documentation/examples are updated for changed settings or contracts ## Final check - [x] This PR addresses one coherent problem without unrelated cleanup - [x] It does not add a speculative API, fallback, or compatibility path - [x] No secrets, local state, model weights, voice material, or restricted assets are included - [x] Third-party notices and provenance are preserved --------- Co-authored-by: Lucas1479 <sli776@aucklanduni.ac.nz>
What and why
Importing
wallpaper.wallpaper_engine_bridgeon macOS currently imports the Windows-only pointer wheel hook at module load time. That hook referencesctypes.WINFUNCTYPE, which is unavailable on non-Windows platforms, so backend startup and test collection fail before wallpaper functionality is used.This PR keeps the type-only import behind
TYPE_CHECKINGand loads the Windows implementation only inside the existingsys.platform == "win32"branch. Windows behavior is unchanged.Linked Issue for product-semantic or public-contract changes: #47
Change class
Owning layer:
wallpaper/wallpaper_engine_bridge.pyUser-visible effect, or
none: The wallpaper bridge and backend can be imported on macOS and Linux; Windows wheel forwarding remains available when enabled.Compatibility or migration impact, or
none: No configuration or migration required.Evidence
Commands and manual journeys run:
python -m pytest -q tests/test_wallpaper_bridge_portability.py tests/test_local_bridge_security.py tests/test_wallpaper_asset_revision.py tests/test_wallpaper_bridge_observability.py— 16 passedpython -m ruff check wallpaper/wallpaper_engine_bridge.py tests/test_wallpaper_bridge_portability.py— passedgit diff --check— passedRelevant Python tests pass
CPU/model-less baseline remains supported
Electron
npm run buildpasses when Electron code changedDependency audit passes when dependencies changed
Before/after screenshots are attached for visible UI changes
Documentation/examples are updated for changed settings or contracts
Final check