5090推理OOM

[whalemjy]

我们在使用readme中的推理示例在5090上进行推理的时候遇到了OOM的问题，请问是需要额外的操作来offload一些用过的模型吗

[whalemjy]

看起来是现在的torch默认安装的是2.9.0，先手动安装2.8.0再执行readme中的安装时也报一些安装失败的情况

[johndpope]

same problem
https://pytorch.org/get-started/previous-versions/
trying this -
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu129

[ckfanzhe]

torch 2.8.0 torchvision 0.23.0 flash_attn 2.8.3 spas_sage_attn 0.1.0
我在WSL中的环境如下，也是同款的OOM，T2V 420P能正常运行 I2V的720P无法运行；额外尝试过torch2.7.0 2.9.1 并且重新编译安装了sage_attn，依旧是OOM

[whx1003]

这个问题我们遇到过, 应该是 torch 版本的问题. 在 torch==2.8.0 下应该可以正常推理

[johndpope]

its gotta be vae decode -

i check my workflow - it was mistakingly using wan_2.1_vae.safetensors - 232mb
we need to use Wan2.1_VAE.pth 434mb
testing now

[johndpope]

success !

[whalemjy]

its gotta be vae decode -

i check my workflow - it was mistakingly using wan_2.1_vae.safetensors - 232mb we need to use Wan2.1_VAE.pth 434mb testing now

We didn't use this pipeline on comfy UI, we deploy it on Ubantu, we've already used proper VAE before this issue, it seems not this problem caused our faliure.

[ckfanzhe]

same problem https://pytorch.org/get-started/previous-versions/ trying this - pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu129

I update the torch's version to 2.8.0 and rebuild the sage_attention python lib, but still have the OOM problem

[whx1003]

same problem https://pytorch.org/get-started/previous-versions/ trying this - pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu129

I update the torch's version to 2.8.0 and rebuild the sage_attention python lib, but still have the OOM problem

Could you provide the full error message?

[ckfanzhe]

============================================================
[16:01:33] [  0.00s] TurboDiffusion I2V Inference
============================================================
[16:01:33] [  0.00s] [I2V-Inference] Device: cuda:0
[16:01:33] [  0.00s] [I2V-Inference] CUDA available: Yes (1 GPU(s) detected)
[16:01:33] [  0.00s] [I2V-Inference]   GPU 0: NVIDIA GeForce RTX 5090 - 31.84GB total VRAM
[16:01:33] [  0.00s] [I2V-Inference] VRAM at start: 1.40GB used, 30.44GB free (of 31.84GB total)
[16:01:33] [  0.00s] [I2V-Inference] Frames: 33, Steps: 4, Resolution: 480p 1:1
[16:01:33] [  0.00s] [I2V-Inference] Boundary: 0.9, Sigma: 200.0, Seed: 42
[16:01:33] [  0.00s] [I2V-Inference] Extracting text embedding from conditioning...
[16:01:33] [  0.00s] [I2V-Inference] Conditioning type: <class 'list'>
[16:01:33] [  0.00s] [I2V-Inference] Conditioning[0] type: <class 'list'>
[16:01:33] [  0.00s] [I2V-Inference] Conditioning[0][0] type: <class 'torch.Tensor'>
[16:01:33] [  0.00s] [I2V-Inference] Text embedding shape: torch.Size([1, 512, 4096])
[16:01:33] [  0.00s] [I2V-Inference] Preparing VAE...
[16:01:33] [  0.00s] [I2V-Inference] Preprocessing input image...
[16:01:33] [  0.01s] [I2V-Inference] Target resolution: 640x640, Latent shape: 9x80x80
[16:01:33] [  0.01s] [I2V-Inference] Frame tensor size: ~0.15GB
[16:01:33] [  0.02s] [I2V-Inference] Encoding image with VAE...
[16:01:33] [  0.02s] [I2V-Inference] VRAM before cleanup: 1.40GB allocated, 1.56GB reserved
[16:01:33] [  0.02s] [I2V-Inference] Unloading all models from GPU...
[16:01:33] [  0.02s] [I2V-Inference] VRAM after cleanup: 1.40GB allocated, 1.56GB reserved
[16:01:33] [  0.02s] [I2V-Inference] Creating frame tensor (1 real frame + 32 zero frames)...
[16:01:33] [  0.06s] [I2V-Inference] Encoding 33 frames at 640x640 resolution...
Moving VAE to cuda:0
Moving VAE to cpu
[16:01:39] [  5.56s] [I2V-Inference] VAE encoding complete (VAE automatically offloaded to CPU)
[16:01:39] [  5.56s] [I2V-Inference] Preparing conditioning...
[16:01:39] [  5.57s] [I2V-Inference] Initializing noise with seed 42...
[16:01:39] [  5.57s] [I2V-Inference] Running dual-expert rCM sampling...
[16:01:39] [  5.57s] [I2V-Inference] Boundary at step 3/4
[16:01:39] [  5.57s] [I2V-Inference] Aggressive VRAM cleanup before loading diffusion model...
[16:01:39] [  5.86s] [I2V-Inference] VRAM before high noise model: 0.02GB allocated, 0.12GB reserved, 31.72GB free
[16:01:39] [  5.86s] [I2V-Inference] Loading high noise model...
[16:01:39] [  0.00s] ⏳ Lazy loading model: TurboWan2.2-I2V-A14B-high-720P-quant.pth...
[16:01:39] [216.51s] [ModelLoader] Loading with official create_model()...
[12-26 16:01:40|INFO|custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/networks/wan2pt2.py:793:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 40
[16:01:40] [216.76s] [ModelLoader] Applying sagesla attention with topk=0.1...
[16:01:40] [216.76s] [ModelLoader] Loading state dict to CPU...
[16:01:58] [235.67s] [ModelLoader] Cleaned 1575 state dict keys
[16:01:58] [235.68s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[16:01:58] [235.71s] [ModelLoader] Loading weights into model...
[16:01:58] [235.73s] [ModelLoader] Model loaded to CPU
[16:01:58] [235.74s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[16:01:58] [235.74s] [ModelLoader] ✓ Successfully loaded model
[16:01:58] [235.74s] [ModelLoader] Model type: Wan2.2-A14B
[16:01:58] [235.74s] [ModelLoader] Attention: sagesla
[16:01:58] [235.74s] [ModelLoader] Quantized: True
[16:01:58] [ 19.24s] ✓ Model loaded in 19.24s: TurboWan2.2-I2V-A14B-high-720P-quant.pth
[16:01:59] [ 25.10s] [I2V-Inference] VRAM after high noise model loaded: 0.02GB allocated, 0.12GB reserved, 31.72GB free
[16:01:59] [ 25.10s] [I2V-Inference] Sampling with high noise model (steps 0-3)...
Sampling steps 0-3:   0%|                                                                                                                                                 | 0/3 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 29553.41 MB usable, 445.90 MB loaded, full load: True
Sampling steps 0-3: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:13<00:00,  4.49s/it]
[16:02:12] [ 38.58s] [I2V-Inference] Offloading high noise model...
[16:02:12] [ 38.58s] [I2V-Inference] Aggressive VRAM cleanup before loading low noise model...
[16:02:13] [ 39.52s] [I2V-Inference] VRAM before low noise model: 13.56GB allocated, 13.62GB reserved, 18.22GB free
[16:02:13] [ 39.52s] [I2V-Inference] Loading low noise model...
[16:02:13] [  0.00s] ⏳ Lazy loading model: TurboWan2.2-I2V-A14B-low-720P-quant.pth...
[16:02:13] [250.16s] [ModelLoader] Loading with official create_model()...
[12-26 16:02:13|INFO|custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/networks/wan2pt2.py:793:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 40
[16:02:13] [250.43s] [ModelLoader] Applying sagesla attention with topk=0.1...
[16:02:13] [250.44s] [ModelLoader] Loading state dict to CPU...
[16:02:52] [289.43s] [ModelLoader] Cleaned 1575 state dict keys
[16:02:52] [289.43s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[16:02:52] [289.47s] [ModelLoader] Loading weights into model...
[16:02:52] [289.50s] [ModelLoader] Model loaded to CPU
[16:02:52] [289.50s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[16:02:52] [289.50s] [ModelLoader] ✓ Successfully loaded model
[16:02:52] [289.50s] [ModelLoader] Model type: Wan2.2-A14B
[16:02:52] [289.50s] [ModelLoader] Attention: sagesla
[16:02:52] [289.50s] [ModelLoader] Quantized: True
[16:02:52] [ 39.34s] ✓ Model loaded in 39.34s: TurboWan2.2-I2V-A14B-low-720P-quant.pth
[16:02:52] [ 78.86s] [I2V-Inference] VRAM after low noise model loaded: 13.56GB allocated, 13.62GB reserved, 18.22GB free
[16:02:52] [ 78.86s] [I2V-Inference] Sampling with low noise model (steps 3-4)...
Sampling steps 3-4:   0%|                                                                                                                                                 | 0/1 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 15686.11 MB usable, 445.90 MB loaded, full load: True
Sampling steps 3-4:   0%|                                                                                                                                                 | 0/1 [00:05<?, ?it/s]
!!! Exception during processing !!! Allocation on device 
Traceback (most recent call last):
  File "/data/ComfyUI/execution.py", line 516, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "/data/ComfyUI/execution.py", line 330, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "/data/ComfyUI/execution.py", line 304, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "/data/ComfyUI/execution.py", line 292, in process_inputs
    result = f(**inputs)
  File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/nodes/turbowan_inference.py", line 414, in generate
    x = rcm_sampler(
  File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/cm_sampler.py", line 77, in rcm_sampler
    v_pred = model(
  File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/lazy_loader.py", line 139, in __call__
    return self._model(*args, **kwargs)
  File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/comfy_native_offload.py", line 118, in __call__
    self._ensure_loaded()
  File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/comfy_native_offload.py", line 85, in _ensure_loaded
    mm.load_model_gpu(self.patcher)
  File "/data/ComfyUI/comfy/model_management.py", line 706, in load_model_gpu
    return load_models_gpu([model])
  File "/data/ComfyUI/comfy/model_management.py", line 701, in load_models_gpu
    loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
  File "/data/ComfyUI/comfy/model_management.py", line 506, in model_load
    self.model_use_more_vram(use_more_vram, force_patch_weights=force_patch_weights)
  File "/data/ComfyUI/comfy/model_management.py", line 536, in model_use_more_vram
    return self.model.partially_load(self.device, extra_memory, force_patch_weights=force_patch_weights)
  File "/data/ComfyUI/comfy/model_patcher.py", line 981, in partially_load
    raise e
  File "/data/ComfyUI/comfy/model_patcher.py", line 978, in partially_load
    self.load(device_to, lowvram_model_memory=current_used + extra_memory, force_patch_weights=force_patch_weights, full_load=full_load)
  File "/data/ComfyUI/comfy/model_patcher.py", line 800, in load
    self.model.to(device_to)
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1371, in to
    return self._apply(convert)
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
    module._apply(fn)
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
    module._apply(fn)
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1022, in _apply
    self._buffers[key] = fn(buf)
  File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1357, in convert
    return t.to(
torch.OutOfMemoryError: Allocation on device 

Got an OOM, unloading all loaded models.
comfyui lumi batcher overwrite task done
Prompt executed in 106.81 seconds

The only log I can paste is from ComfyUI; directly executing the TurboDiffusion code causes the entire WSL to be killed.

[johndpope]

i follow instructions to the t - need to add psutil here

git submodule update --init --recursive 
conda create -n turbodiffusion python=3.12
conda activate turbodiffusion
pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu130

pip install psutil
pip install turbodiffusion --no-build-isolation
pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation

[whx1003]

============================================================
[16:01:33] [ 0.00s] TurboDiffusion I2V Inference

[16:01:33] [ 0.00s] [I2V-Inference] Device: cuda:0
[16:01:33] [ 0.00s] [I2V-Inference] CUDA available: Yes (1 GPU(s) detected)
[16:01:33] [ 0.00s] [I2V-Inference] GPU 0: NVIDIA GeForce RTX 5090 - 31.84GB total VRAM
[16:01:33] [ 0.00s] [I2V-Inference] VRAM at start: 1.40GB used, 30.44GB free (of 31.84GB total)
[16:01:33] [ 0.00s] [I2V-Inference] Frames: 33, Steps: 4, Resolution: 480p 1:1
[16:01:33] [ 0.00s] [I2V-Inference] Boundary: 0.9, Sigma: 200.0, Seed: 42
[16:01:33] [ 0.00s] [I2V-Inference] Extracting text embedding from conditioning...
[16:01:33] [ 0.00s] [I2V-Inference] Conditioning type: <class 'list'>
[16:01:33] [ 0.00s] [I2V-Inference] Conditioning[0] type: <class 'list'>
[16:01:33] [ 0.00s] [I2V-Inference] Conditioning[0][0] type: <class 'torch.Tensor'>
[16:01:33] [ 0.00s] [I2V-Inference] Text embedding shape: torch.Size([1, 512, 4096])
[16:01:33] [ 0.00s] [I2V-Inference] Preparing VAE...
[16:01:33] [ 0.00s] [I2V-Inference] Preprocessing input image...
[16:01:33] [ 0.01s] [I2V-Inference] Target resolution: 640x640, Latent shape: 9x80x80
[16:01:33] [ 0.01s] [I2V-Inference] Frame tensor size: ~0.15GB
[16:01:33] [ 0.02s] [I2V-Inference] Encoding image with VAE...
[16:01:33] [ 0.02s] [I2V-Inference] VRAM before cleanup: 1.40GB allocated, 1.56GB reserved
[16:01:33] [ 0.02s] [I2V-Inference] Unloading all models from GPU...
[16:01:33] [ 0.02s] [I2V-Inference] VRAM after cleanup: 1.40GB allocated, 1.56GB reserved
[16:01:33] [ 0.02s] [I2V-Inference] Creating frame tensor (1 real frame + 32 zero frames)...
[16:01:33] [ 0.06s] [I2V-Inference] Encoding 33 frames at 640x640 resolution...
Moving VAE to cuda:0
Moving VAE to cpu
[16:01:39] [ 5.56s] [I2V-Inference] VAE encoding complete (VAE automatically offloaded to CPU)
[16:01:39] [ 5.56s] [I2V-Inference] Preparing conditioning...
[16:01:39] [ 5.57s] [I2V-Inference] Initializing noise with seed 42...
[16:01:39] [ 5.57s] [I2V-Inference] Running dual-expert rCM sampling...
[16:01:39] [ 5.57s] [I2V-Inference] Boundary at step 3/4
[16:01:39] [ 5.57s] [I2V-Inference] Aggressive VRAM cleanup before loading diffusion model...
[16:01:39] [ 5.86s] [I2V-Inference] VRAM before high noise model: 0.02GB allocated, 0.12GB reserved, 31.72GB free
[16:01:39] [ 5.86s] [I2V-Inference] Loading high noise model...
[16:01:39] [ 0.00s] ⏳ Lazy loading model: TurboWan2.2-I2V-A14B-high-720P-quant.pth...
[16:01:39] [216.51s] [ModelLoader] Loading with official create_model()...
[12-26 16:01:40|INFO|custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/networks/wan2pt2.py:793:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 40
[16:01:40] [216.76s] [ModelLoader] Applying sagesla attention with topk=0.1...
[16:01:40] [216.76s] [ModelLoader] Loading state dict to CPU...
[16:01:58] [235.67s] [ModelLoader] Cleaned 1575 state dict keys
[16:01:58] [235.68s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[16:01:58] [235.71s] [ModelLoader] Loading weights into model...
[16:01:58] [235.73s] [ModelLoader] Model loaded to CPU
[16:01:58] [235.74s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[16:01:58] [235.74s] [ModelLoader] ✓ Successfully loaded model
[16:01:58] [235.74s] [ModelLoader] Model type: Wan2.2-A14B
[16:01:58] [235.74s] [ModelLoader] Attention: sagesla
[16:01:58] [235.74s] [ModelLoader] Quantized: True
[16:01:58] [ 19.24s] ✓ Model loaded in 19.24s: TurboWan2.2-I2V-A14B-high-720P-quant.pth
[16:01:59] [ 25.10s] [I2V-Inference] VRAM after high noise model loaded: 0.02GB allocated, 0.12GB reserved, 31.72GB free
[16:01:59] [ 25.10s] [I2V-Inference] Sampling with high noise model (steps 0-3)...
Sampling steps 0-3: 0%| | 0/3 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 29553.41 MB usable, 445.90 MB loaded, full load: True
Sampling steps 0-3: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:13<00:00, 4.49s/it]
[16:02:12] [ 38.58s] [I2V-Inference] Offloading high noise model...
[16:02:12] [ 38.58s] [I2V-Inference] Aggressive VRAM cleanup before loading low noise model...
[16:02:13] [ 39.52s] [I2V-Inference] VRAM before low noise model: 13.56GB allocated, 13.62GB reserved, 18.22GB free
[16:02:13] [ 39.52s] [I2V-Inference] Loading low noise model...
[16:02:13] [ 0.00s] ⏳ Lazy loading model: TurboWan2.2-I2V-A14B-low-720P-quant.pth...
[16:02:13] [250.16s] [ModelLoader] Loading with official create_model()...
[12-26 16:02:13|INFO|custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/networks/wan2pt2.py:793:enable_selective_checkpoint] Enable selective checkpoint with mm_only, for every 1 blocks. Total blocks: 40
[16:02:13] [250.43s] [ModelLoader] Applying sagesla attention with topk=0.1...
[16:02:13] [250.44s] [ModelLoader] Loading state dict to CPU...
[16:02:52] [289.43s] [ModelLoader] Cleaned 1575 state dict keys
[16:02:52] [289.43s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[16:02:52] [289.47s] [ModelLoader] Loading weights into model...
[16:02:52] [289.50s] [ModelLoader] Model loaded to CPU
[16:02:52] [289.50s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[16:02:52] [289.50s] [ModelLoader] ✓ Successfully loaded model
[16:02:52] [289.50s] [ModelLoader] Model type: Wan2.2-A14B
[16:02:52] [289.50s] [ModelLoader] Attention: sagesla
[16:02:52] [289.50s] [ModelLoader] Quantized: True
[16:02:52] [ 39.34s] ✓ Model loaded in 39.34s: TurboWan2.2-I2V-A14B-low-720P-quant.pth
[16:02:52] [ 78.86s] [I2V-Inference] VRAM after low noise model loaded: 13.56GB allocated, 13.62GB reserved, 18.22GB free
[16:02:52] [ 78.86s] [I2V-Inference] Sampling with low noise model (steps 3-4)...
Sampling steps 3-4: 0%| | 0/1 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 15686.11 MB usable, 445.90 MB loaded, full load: True
Sampling steps 3-4: 0%| | 0/1 [00:05<?, ?it/s]
!!! Exception during processing !!! Allocation on device
Traceback (most recent call last):
File "/data/ComfyUI/execution.py", line 516, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "/data/ComfyUI/execution.py", line 330, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "/data/ComfyUI/execution.py", line 304, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/data/ComfyUI/execution.py", line 292, in process_inputs
result = f(**inputs)
File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/nodes/turbowan_inference.py", line 414, in generate
x = rcm_sampler(
File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/turbodiffusion_vendor/rcm/cm_sampler.py", line 77, in rcm_sampler
v_pred = model(
File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/lazy_loader.py", line 139, in call
return self._model(*args, **kwargs)
File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/comfy_native_offload.py", line 118, in call
self._ensure_loaded()
File "/data/ComfyUI/custom_nodes/Comfyui_turbodiffusion/utils/comfy_native_offload.py", line 85, in _ensure_loaded
mm.load_model_gpu(self.patcher)
File "/data/ComfyUI/comfy/model_management.py", line 706, in load_model_gpu
return load_models_gpu([model])
File "/data/ComfyUI/comfy/model_management.py", line 701, in load_models_gpu
loaded_model.model_load(lowvram_model_memory, force_patch_weights=force_patch_weights)
File "/data/ComfyUI/comfy/model_management.py", line 506, in model_load
self.model_use_more_vram(use_more_vram, force_patch_weights=force_patch_weights)
File "/data/ComfyUI/comfy/model_management.py", line 536, in model_use_more_vram
return self.model.partially_load(self.device, extra_memory, force_patch_weights=force_patch_weights)
File "/data/ComfyUI/comfy/model_patcher.py", line 981, in partially_load
raise e
File "/data/ComfyUI/comfy/model_patcher.py", line 978, in partially_load
self.load(device_to, lowvram_model_memory=current_used + extra_memory, force_patch_weights=force_patch_weights, full_load=full_load)
File "/data/ComfyUI/comfy/model_patcher.py", line 800, in load
self.model.to(device_to)
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1371, in to
return self._apply(convert)
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
module._apply(fn)
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
module._apply(fn)
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 930, in _apply
module._apply(fn)
[Previous line repeated 2 more times]
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1022, in _apply
self._buffers[key] = fn(buf)
File "/data/miniconda3/envs/self_forcing/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1357, in convert
return t.to(
torch.OutOfMemoryError: Allocation on device

Got an OOM, unloading all loaded models.
comfyui lumi batcher overwrite task done
Prompt executed in 106.81 seconds
The only log I can paste is from ComfyUI; directly executing the TurboDiffusion code causes the entire WSL to be killed.

According to our tests, TurboDiffusion uses nearly all available VRAM (around 98%) on an RTX 5090 when generating I2V videos. I noticed the following log message:

VRAM at start: 1.40GB used, 30.44GB free (of 31.84GB total)

This suggests that some VRAM might already be reserved by ComfyUI or other programs, which could lead to the OOM issue. I recommend reporting this under the ComfyUI_TurboDiffusion repository.

[johndpope]

i add this install script - #53

i have create a staged pipeline to spit out t5 embedings 4mb (reduces 11gb) then further dit high/low pass unloadings -
this only helps for batches though.