Skip to content

Commit e0553aa

Browse files
fix: address CodeRabbit review findings — MPS cache cleanup, MLX unload thread-safety
Two follow-ups from the automated review on this PR: 1. backend/backends/base.py: empty_device_cache() only handled "cuda" and "xpu", never "mps" — so once the previous commit enabled MPS for the PyTorch Qwen3-TTS/CustomVoice engines, switching model sizes on Apple Silicon would leak MPS-cached memory (torch.mps.empty_cache() never got called). Added the missing branch. 2. backend/backends/qwen_custom_voice_backend.py: unload_model() had its own inline `if torch.cuda.is_available(): torch.cuda.empty_cache()` instead of using the shared empty_device_cache() helper — meant it never freed MPS memory either, same bug as #1 but duplicated instead of shared. Switched to the shared helper (matches pytorch_backend.py's existing pattern). 3. backend/backends/mlx_backend.py: load_model_async()'s unload-on- model-size-change call went straight to self.unload_model() instead of routing through _run_on_mlx_thread like every other MLX call in this file (the whole point of the earlier commit in this PR). Fixed for consistency with the thread-pinning fix. Verified on M4: 0.6B -> generate still works post-changes (regression check), torch.mps.empty_cache() confirmed callable without error in this environment.
1 parent 7cfa6b8 commit e0553aa

3 files changed

Lines changed: 11 additions & 7 deletions

File tree

backend/backends/base.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -171,17 +171,19 @@ def check_cuda_compatibility() -> tuple[bool, str | None]:
171171

172172
def empty_device_cache(device: str) -> None:
173173
"""
174-
Free cached memory on the given device (CUDA or XPU).
174+
Free cached memory on the given device (CUDA, XPU, or MPS).
175175
176-
Backends should call this after unloading models so VRAM is returned
177-
to the OS.
176+
Backends should call this after unloading models so VRAM/unified
177+
memory is returned to the OS.
178178
"""
179179
import torch
180180

181181
if device == "cuda" and torch.cuda.is_available():
182182
torch.cuda.empty_cache()
183183
elif device == "xpu" and hasattr(torch, "xpu"):
184184
torch.xpu.empty_cache()
185+
elif device == "mps" and hasattr(torch, "mps"):
186+
torch.mps.empty_cache()
185187

186188

187189
def manual_seed(seed: int, device: str) -> None:

backend/backends/mlx_backend.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,9 +93,11 @@ async def load_model_async(self, model_size: Optional[str] = None):
9393
if self.model is not None and self._current_model_size == model_size:
9494
return
9595

96-
# Unload existing model if different size requested
96+
# Unload existing model if different size requested — routed
97+
# through the same dedicated thread as load/generate so it stays
98+
# serialized with any in-flight MLX call (see _run_on_mlx_thread).
9799
if self.model is not None and self._current_model_size != model_size:
98-
self.unload_model()
100+
await _run_on_mlx_thread(self.unload_model)
99101

100102
# Run blocking load on the dedicated MLX thread
101103
await _run_on_mlx_thread(self._load_model_sync, model_size)

backend/backends/qwen_custom_voice_backend.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
from .base import (
2626
is_model_cached,
2727
get_torch_device,
28+
empty_device_cache,
2829
combine_voice_prompts as _combine_voice_prompts,
2930
model_load_progress,
3031
)
@@ -127,8 +128,7 @@ def unload_model(self) -> None:
127128
self.model = None
128129
self._current_model_size = None
129130

130-
if torch.cuda.is_available():
131-
torch.cuda.empty_cache()
131+
empty_device_cache(self.device)
132132

133133
logger.info("Qwen CustomVoice unloaded")
134134

0 commit comments

Comments
 (0)