when is Consumer GPU support coming??

[mrsuperwealthy]

i want to run this locally and efficiently

[zcfrank1st]

+1

[Dai-Wenxun]

+1

[lailai4]

+1

[Dai-Wenxun]

@mrsuperwealthy @zcfrank1st @lailai4 Update on this: RTX 5090 support lands this week.

The kernel side is already de-risked — the 5090 is the same sm_120 (Blackwell) architecture as the RTX PRO 6000 we run in production, where the current stack (FP8 + fused ops + SageAttention, all sm_120-native) sustains 480p @ 24 fps / 720p @ 16 fps real-time. The remaining piece — fitting the 32 GB memory budget (vs 96 GB) — is being finalized now and is expected on main within the week.

Until then, you can run it "locally" in spirit — live browser demo, zero setup: https://huggingface.co/spaces/wxDai/joyai-video-edit. One caveat: the demo runs 480p @ 24 fps to stay real-time, and the model is trained at 720p — so expect somewhat lower visual/editing quality there than a local 720p deployment. The new 0811 checkpoint and the overhauled deployment guide (DEPLOYMENT.md, bash deploy/run_server.sh as the single entry point) are already on main.

[Dai-Wenxun]

@mrsuperwealthy @zcfrank1st @lailai4 Delivered — consumer GPU support is on main. A single GeForce RTX 5090 (32 GB) now runs the full real-time streaming pipeline at 840 × 480 @ 24 FPS.

Launch:

JOYOMNI_SAGE_ATTN=1 JOYOMNI_FP8_FAST_ACCUM=1 JOYOMNI_LOW_VRAM=1 \
bash deploy/run_server.sh

What the three switches do:

• JOYOMNI_LOW_VRAM=1 — the 16.3B DiT is quantized to FP8 block-by-block during loading (the full bf16 model never materializes in VRAM), and the text encoder runs with CPU offload. Measured at 480p24: ~21.5 GiB resident / ~28 GiB peak — comfortable on 32 GB.
• JOYOMNI_SAGE_ATTN=1 — INT8 SageAttention. GeForce runs fp32-accumulate SDPA at half rate, so this matters far more on the 5090 than on the datacenter cards.
• JOYOMNI_FP8_FAST_ACCUM=1 — FP8 GEMMs accumulate in fp16 (same GeForce half-rate story).

Full setup — env, prebuilt wheels, per-GPU launch commands, knob table — is in the Deployment Guide (RTX 5090 section near the end). 480p @ 24 FPS is the validated real-time config on the 5090; real-time 720p still needs the bigger cards.

If anything breaks on your setup, please open a new issue with the log. Enjoy! 🚀