Enable AVXVNNI microkernels for mobile platforms #7828
Conversation
src/configs/gemm-config.c
Outdated
| @@ -2865,7 +2865,7 @@ static void init_qdu8_f16_qc8w_gemm_config(void) { | |||
| } else | |||
| #endif | |||
| #if XNN_ENABLE_AVXVNNI | |||
| if (!XNN_PLATFORM_MOBILE && hardware_config->use_x86_avxvnni) { | |||
| if (hardware_config->use_x86_avxvnni) { | |||
There was a problem hiding this comment.
the meaning of 'XNN_PLATFORM_MOBILE' is really iphone and android (arm) devices using intel for emulating iphone when running on x86 mac, or x86 on linux for android studio, to mimic an arm device.
the mac will never have avxvnni... the last x86 mac was broadwell.
a linux or windows development machine could have avxvnni, but its not critical to use kernels for debugging
They are disabled for practical reasons - an old NDK / compiler wont have new instruction sets.
There was a problem hiding this comment.
Alderlake should already be enabled for Windows, Linux, Chromebooks etc.
Its just not enabled for iOS or Android
Can you confirm?
There was a problem hiding this comment.
Yes for Linux, Windows and Chromebooks it's enabled, but Android on x86 is also available (like Celadon which is used for Cloud Gaming, Smart Retail etc).
Celadon - https://www.intel.com/content/www/us/en/developer/topic-technology/open/celadon/overview.html
So avxvnni kernels for Android on x86 should be enabled to leverage the benefits of this instruction set.
There was a problem hiding this comment.
oh cool. For android it makes sense. At the time it was because iOS and Android are really for ARM devices and the x86 builds were just for the 'Simulator' and Android Studio.
In the past there are old NDK and ios tools that dont have avx512 or modern x86 instruction sets.
You did avxvnni, which is relatively new; newer than avx512 vnni.
So I agree we should try removing some MOBILE if statements, but proceed with caution
in src/xnnpack/common.h
// Mobile build x86 versions for debugging
#if XNN_PLATFORM_ANDROID || XNN_PLATFORM_IOS
#define XNN_PLATFORM_MOBILE 1
#else
#define XNN_PLATFORM_MOBILE 0
#endif
XNN_PLATFORM_MOBILE is set for iphone as well as android, and removing the if from avxvnni enables it for iphone, which is likely problematic, because the last x86 macos was Intel Broadwell based, and did not have avxvnni or avx512 vnni.
So instead, on any x86 kernels you want to try enabling for android, change
XNN_PLATFORM_MOBILE to XNN_PLATFORM_IOS ?
The macro XNN_ENABLE_AVXVNNI can be disabled. With bazel the option
-define=xnn_enable_avxvnni=false
for android if an old ndk doesnt build, avxvnni can be disabled, either with the bazel build flag, or the CMakeLists.txt check for compiler version.
The last old NDK we supported was 21, which has gcc 9 and clang 9, which do not have avxvnni.
There was a problem hiding this comment.
Thanks for update. As per your suggestion I have updated XNN_PLATFORM_MOBILE to XNN_PLATFORM_IOS so that it remain disabled for iPhone.
There was a problem hiding this comment.
I have checked that in CMakeList.txt there are already checks available for compiler version which does not support avxvnni / avxvnniint8.
in CMakeList.txt
OPTION(XNNPACK_ENABLE_AVXVNNI "Build XNNPACK with AVX-VNNI micro-kernels" ON)
IF(CMAKE_C_COMPILER_ID STREQUAL "GNU")
IF(CMAKE_C_COMPILER_VERSION VERSION_LESS "11")
SET(XNNPACK_ENABLE_AVXVNNI OFF)
ENDIF()
ELSEIF(CMAKE_C_COMPILER_ID STREQUAL "Clang")
IF(CMAKE_C_COMPILER_VERSION VERSION_LESS "15")
SET(XNNPACK_ENABLE_AVXVNNI OFF)
ENDIF()
ENDIF()
.
.
IF(CMAKE_C_COMPILER_ID STREQUAL "GNU")
IF(CMAKE_C_COMPILER_VERSION VERSION_LESS "17")
SET(XNNPACK_ENABLE_AVXVNNIINT8 OFF)
ENDIF()
ELSEIF(CMAKE_C_COMPILER_ID STREQUAL "Clang")
IF(CMAKE_C_COMPILER_VERSION VERSION_LESS "17")
SET(XNNPACK_ENABLE_AVXVNNIINT8 OFF)
ENDIF()
soniravi31
force-pushed
the
enable_mobile_avxvnni
branch
2 times, most recently
from
19a57ca to
478d3c4
Compare
AlderLake onwards x86 mobile platform supports avxvnni Signed-off-by: Ravi Kumar Soni <[email protected]>
AlderLake onwards x86 mobile platform supports avxvnni