Enabling GPU acceleration failed on aarch64-oe-linux-gcc11.2

[jianhuiliao]

An error occurred due to the use of the pre-built GPU accelerator for linux arm64 platform.

This might be because the libLiteRtWebGpuAccelerator.so library was compiled with dotprod enabled via hardcoding.

When will you open-source the GPU accelerator plugin code so that the community can build with suitable options?

Error logs:
==19745==
INFO: [/workspace/qairt/liteRT/tensorflow/LiRT_src/litert/core/environment.cc:29] Creating LiteRT environment with options
INFO: [/workspace/qairt/liteRT/tensorflow/LiRT_src/litert/runtime/accelerators/auto_registration.cc:149] Loading GPU accelerator(libLiteRtGpuAccelerator.so).
INFO: [/workspace/qairt/liteRT/tensorflow/LiRT_src/litert/runtime/accelerators/auto_registration.cc:149] Loading GPU accelerator(libLiteRtWebGpuAccelerator.so).
FATAL ERROR: This binary was compiled with dotprod enabled, but this feature is not available on this processor (go/sigill-fail-fast).
==19745==
==19745== Process terminating with default action of signal 4 (SIGILL): dumping core
==19745== at 0x4C047F8: __pthread_kill_implementation (pthread_kill.c:44)
==19745== by 0x4C04873: __pthread_kill_internal (pthread_kill.c:78)
==19745== by 0x4C0489F: pthread_kill@@GLIBC_2.34 (pthread_kill.c:89)
==19745== by 0x4BBD78F: raise (raise.c:26)
==19745== by 0x600C933: ??? (in /data/local/tmp/tflite/libLiteRtWebGpuAccelerator.so)
==19745== by 0x40057F7: _dl_init (dl-init.c:106)
==19745== by 0x400B97F: call_dl_init (dl-open.c:485)
==19745== by 0x4CB0D53: _dl_catch_exception (dl-error-skeleton.c:182)
==19745== by 0x400BF0F: dl_open_worker (dl-open.c:808)
==19745== by 0x4CB0CFF: _dl_catch_exception (dl-error-skeleton.c:208)
==19745== by 0x400C1FB: _dl_open (dl-open.c:884)
==19745== by 0x4BFEB57: dlopen_doit (dlopen.c:56)
==19745==
==19745== HEAP SUMMARY:
==19745== in use at exit: 90,910 bytes in 110 blocks
==19745== total heap usage: 257 allocs, 147 frees, 104,849 bytes allocated
==19745==
==19745== LEAK SUMMARY:
==19745== definitely lost: 0 bytes in 0 blocks
==19745== indirectly lost: 0 bytes in 0 blocks
==19745== possibly lost: 3,896 bytes in 2 blocks
==19745== still reachable: 87,014 bytes in 108 blocks
==19745== suppressed: 0 bytes in 0 blocks
==19745== Rerun with --leak-check=full to see details of leaked memory
==19745==
==19745== For lists of detected and suppressed errors, rerun with: -s
==19745== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Illegal instruction

[sourcelite]

Hi @jianhuiliao, welcome here thanks for logs. it occurs on arm64 when binary is bult for moden arch features like one you mentioned dotprod that arent present on all hardwares.

I did some root cause analysis here is for your reference - log is giving clarity

FATAL ERROR: This binary was compiled with dotprod enabled, but this feature is not available on this processor (go/sigill-fail-fast)

Instruction mismatch, ibLiteRtWebGpuAccelerator.so this prebuilt is compiled with ARMv8.2-A Dot Product extension enabled usually via -march=armv8.2-a+dotprod or any other similar flags. Your processor does not support the "FEAT_DotProd" instruction, this caused the CPU raise a CPU - SIGILL (illegal/invalid instruction), it caused the the crash. I will look further in this issue...

[sourcelite]

When will you open-source the GPU accelerator plugin code so that the community can build with suitable options ?

while I will check if possible to opensource plugin code GPU accelerator (ML Drift), I will like to suggest you some specific immediate workaround for you to get unblocked -

1. I understand that no-one can build the GPU plugin ourselves, but for SIGILL crash stopage I suggest you to disable the WebGPU Auto-Registration - if you dont need the WebGPU then simply remove the libLiteRtWebGpuAccelerator.so from lib path. LiteRT will automatically fall back to the std lib - libLiteRtGpuAccelerator.so CPU or opencl.
2. secondly explicit accelerator selection - instead of letting the auto_registration.cc load everything - you can specifiy the hardware during env creation for avoiding the loader touching incompatible libs.

how to do it ? - suggested sample -

// Explicitly avoid loading the WebGPU plugin if not on compatible hardware
litert::EnvironmentOptions options;
options.AddAccelerator(litert::Accelerator::kCpu); 
// Only add GPU if you can verify it's the standard OpenCL version
auto env = litert::Environment::Create(options);

hope this will help you avoid the crashes. thanks!

[sourcelite]

This issue is similar to #5721
(query is related to making GPU accelerator code open sourced anytime soon)

[github-actions]

Marking this issue as stale since it has been open for 7 days with no activity. This issue will be closed if no further activity occurs.

[github-actions]

This issue was closed because it has been inactive for 14 days. Please post a new issue if you need further assistance. Thanks!