-
Does anyone know which docker image supports AMX? These were the images that were pushed to docker hub, it doesn't say AMX. What does Native and Fancy tags mean? TAG |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
According to [1], AMX docker image is currently unavailable. And according to [2], NATIVE seems to use CPU instruction capacities of compiling host, and FANCY seems enables extra AVX-512 instructions (compare with [3]) and perhaps performance boost. 1: https://www.reddit.com/r/LocalLLaMA/comments/1ka94qx/qwen_3_ktransformers_03_amx_ai_workstationpc/ |
Beta Was this translation helpful? Give feedback.
According to [1], AMX docker image is currently unavailable.
And according to [2], NATIVE seems to use CPU instruction capacities of compiling host, and FANCY seems enables extra AVX-512 instructions (compare with [3]) and perhaps performance boost.
1: https://www.reddit.com/r/LocalLLaMA/comments/1ka94qx/qwen_3_ktransformers_03_amx_ai_workstationpc/
2: https://github.com/kvcache-ai/ktransformers/blob/7530491f5b08a4e536a31fc5668e19ddb8abc737/csrc/ktransformers_ext/CMakeLists.txt
3: https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512