would intel cpu like xeon be supported?

[leiwen83]

XEON seems a good platform to host large model

[zeux]

I've never tested on Xeons with good amount of fast memory, but in theory they should work well - this project has AVX2 kernels written to support CPU inference and uses OMP for multi-threading. There might be some tuning that would be required if the target HW has a lot of memory bandwidth available though, as CPU perf was only tested on desktop systems with a moderate amount of memory bandwidth (eg ~50 GB/s on Zen 4 platform).