How much VRAM is needed to run this model?

[walkingwithGod2017]

Hello, thank you very much for your research work. The demonstration effect looks excellent. May I ask approximately how much VRAM is needed to run this model currently? Can the RTX 5090 be used for video editing? Looking forward to your reply!

[EasonXiao-888]

Thank you for your interest! The project currently requires 48GB of VRAM. However, as noted in the first item on our TODO list, we are actively working on optimizing it for the RTX 5090. Please stay tuned for updates.

[walkingwithGod2017]

Thank you for your interest! The project currently requires 48GB of VRAM. However, as noted in the first item on our TODO list, we are actively working on optimizing it for the RTX 5090. Please stay tuned for updates.

Thank you, I am very much looking forward to your future research progress.

[coder-bossin]

May I ask if 48G of VRAM is enough? I'm using a single L40s GPU, and the service crashes with an OOM error as soon as I start it.

[Dai-Wenxun]

@walkingwithGod2017 To close the loop on your 5090 question: yes — RTX 5090 support lands this week. It shares the sm_120 architecture with the RTX PRO 6000 we already run (480p @ 24 fps / 720p @ 16 fps real-time), so the kernels are proven; what's being finalized is fitting the 32 GB memory budget.

@coder-bossin About the L40S OOM — the 48 GB figure is essentially the weights floor (16.3B DiT in bf16 ≈ 33 GB + 7B text encoder ≈ 15 GB), before activations, KV cache and CUDA-graph buffers. So a single 48 GB card OOMs on the full real-time pipeline as you observed; the validated real-time configs today are 96 GB-class (RTX PRO 6000 / B200). The low-VRAM work landing this week (5090 included) is exactly what changes this picture for smaller cards.

Meanwhile, anyone can try it with zero hardware — live browser demo: https://huggingface.co/spaces/wxDai/joyai-video-edit (new 0811 checkpoint; deployment guide fully refreshed: DEPLOYMENT.md, bash deploy/run_server.sh is the single entry point). Note the demo runs 480p @ 24 fps for real-time while the model is trained at 720p, so it undersells the quality a local 720p run gives.

[Dai-Wenxun]

@walkingwithGod2017 Closing the loop for good: yes — the RTX 5090 works now. As of today, main runs real-time 840 × 480 @ 24 FPS on a single RTX 5090 (32 GB):

JOYOMNI_SAGE_ATTN=1 JOYOMNI_FP8_FAST_ACCUM=1 JOYOMNI_LOW_VRAM=1 \
bash deploy/run_server.sh

So the VRAM answer at the top of this thread is out of date: with JOYOMNI_LOW_VRAM=1, the DiT loads straight to FP8, block by block (the ~33 GB bf16 model never materializes in VRAM) and the text encoder offloads to CPU — measured ~21.5 GiB resident / ~28 GiB peak at 480p24. 32 GB is now enough. Details in the Deployment Guide (RTX 5090 section).

@coder-bossin The same path changes the picture for your L40S memory-wise (28 GiB peak ≪ 48 GB). One honest caveat: everything in the guide is built and validated against Blackwell (sm_100/sm_120); the L40S is Ada (sm_89), which we haven't validated — the memory budget should now fit, but I can't vouch for the FP8/attention kernel paths there, let alone real-time throughput. If you try it, please report back in a new issue.