关于速度问题

[ganjunhong]

非常感谢博主大佬的项目，在测试代码和跟博主大佬的演示站对比，发现本地的速度非常慢。知不道是不是我的配置不对。用的是：ChineseErrorCorrector3-4B。python version=3.10
2025-09-24 19:46:27.723 | DEBUG | pycorrector.gpt.gpt_model:init:105 - Device: cuda
Loading checkpoint shards: 100%|██████████| 2/2 [04:11<00:00, 125.87s/it]
Some parameters are on the meta device because they were offloaded to the disk and cpu.
-hello man
2025-09-24 19:50:45.762 | DEBUG | pycorrector.gpt.gpt_corrector:init:35 - Loaded model: premodel/ChineseErrorCorrector3-4B, spend: 258.190 s.
Generating outputs: 0%| | 0/1 [00:00<?, ?it/s]The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Generating outputs: 100%|██████████| 1/1 [10:29<00:00, 629.24s/it]

[shibing624]

看着没问题的，你是不是gpu显存不足？我看offload到cpu了。我这里用云上的H20部署的。

[ganjunhong]

谢谢大佬回复。这个您这边跑起来大概要多少显存呢。我是3060的12G

[shibing624]

3B模型建议8G以上就可以运行，要求速度建议用3090以上卡。