torch.OutOfMemoryError: HIP out of memory.

[pan17]

使用3S复刻功能推理几十个字就爆显存了。

显卡：AMD 6800xt
系统：unbuntu 24.04.2 LTS
pytorch: 2.6.0 for rocm6.2.4

torch.OutOfMemoryError: HIP out of memory. Tried to allocate 5.48 GiB. GPU 0 has a total capacity of 15.98 GiB of which 4.96 GiB is free. Of the allocated memory 10.43 GiB is allocated by PyTorch, and 240.40 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_HIP_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[lovewater]

试试流式输出+TRT，可以节约很多显存。另外想问问 6800XT推理的速度如何？

[pan17]

试试流式输出+TRT，可以节约很多显存。另外想问问 6800XT推理的速度如何？

部署的时候不小心改了某个参数导致的，重新部署显存占用就稳定在30%左右，看看后面能不能复现。
推理速度的话用webui.py运行0.5b,三秒复刻，非流式推理，生成1:31分的音频花了1:15分
100%|█████████████████████████████████████████████| 5/5 [01:15<00:00, 15.04s/it]