NUMA需双倍内存问题的优化的疑问

[Moeologist]

Also we want to make further use of our two NUMA nodes on Xeon Gold cpu. To avoid the cost of data transfer between nodes, we "copy" the critical matrix on both nodes which takes more memory consumption but accelerates the prefill and decoding process. But this method takes huge memory and slow when loading weights, So be patient when loading and monitor the memory usage. We are going to optimize this huge memory overhead. Stay tuned~

据文档提到了两个NUMA节点都拷贝内存是可以优化的，不知道优化思路是什么。想请教下
类似张量并行？

[Azure-Tang]

✅

[springnative]

我猜应该是内存速度的问题， cpu 访问自己的内存快，本来带宽就是瓶颈，再交叉访问速度更低了