Overflowing to shared GPU memory even a little bit caused extreme slowdown

[NeverOccurs]

I use 4090. As title described, when dedicated GPU used up and shared GPU memory is used the generation becomes extremely slow. Even the system becomes slow and killing the job becomes slow as well. This usually happens at ksampler or VAE decode node. I tried disabled nvidia system fall back and things like that they all didn't work. I once used blockswap for this but now this node is gone I don't know what can be done to address this.