[Build] how to build onnxruntime with openvino EP for android

[junruizh2021]

Describe the issue

I want to use openvino as Execution Providers to compile onnxruntime, but need to execute on android x86 device. The compilation command I currently use is as follows:

$ ./build.sh --android --android_sdk_path /home/user/sdk --android_ndk_path /home/junruizh/sdk/ndk/27.0.11718014 --android_abi x86_64 --android_api 34 --config RelWithDebInfo --use_openvino GPU --build_shared_lib --cmake_extra_defines OpenVINO_DIR=/home/user/openvino-android/openvino-install-android/runtime/cmake

The openvino_dir specified here is the installation path of an Android version of openvino.

During the compilation process of onnxruntime, I encountered this error:

CMake Error at onnxruntime_providers_openvino.cmake:49 (add_dependencies):
  The dependency target "onnxruntime_providers_shared" of target
  "onnxruntime_providers_openvino" does not exist.
Call Stack (most recent call first):
  onnxruntime_providers.cmake:143 (include)
  CMakeLists.txt:1768 (include)


-- Generating done (0.3s)
CMake Warning:
  Manually-specified variables were not used by the project:

    ANDROID_MIN_SDK
    onnxruntime_USE_OPENVINO_DEVICE

Urgency

No response

Target platform

x86

Build script

$ ./build.sh --android --android_sdk_path /home/user/sdk --android_ndk_path /home/junruizh/sdk/ndk/27.0.11718014 --android_abi x86_64 --android_api 34 --config RelWithDebInfo --use_openvino GPU --build_shared_lib --cmake_extra_defines OpenVINO_DIR=/home/user/openvino-android/openvino-install-android/runtime/cmake

Error / output

CMake Error at onnxruntime_providers_openvino.cmake:49 (add_dependencies):
The dependency target "onnxruntime_providers_shared" of target
"onnxruntime_providers_openvino" does not exist.
Call Stack (most recent call first):
onnxruntime_providers.cmake:143 (include)
CMakeLists.txt:1768 (include)

Visual Studio Version

No response

GCC / Compiler Version

No response

[skottmckay]

It's not a currently supported combination. You could try removing the line that excludes the creation of onnxruntime_providers_shared on Anrdoid as a first step.

onnxruntime/cmake/onnxruntime_providers_cpu.cmake

Line 218 in 2d05c4b

AND NOT CMAKE_SYSTEM_NAME STREQUAL "Android"

[junruizh2021]

Thanks. I'll try it.

[junruizh2021]

@skottmckay As you suggestion, I deleted the cmake Line 218. But I encountered a lot of variable errors, as shown below:

/home/junruizh/onnxruntime/onnxruntime/core/providers/openvino/backend_utils.cc:113:25: error: implicit conversion loses integer precision: 'size_type' (aka 'unsigned long') to 'int' [-Werror,-Wshorten-64-to-32]
  113 |   int pos = output_name.find("/");
      |       ~~~   ~~~~~~~~~~~~^~~~~~~~~
1 error generated.
gmake[2]: *** [CMakeFiles/onnxruntime_providers_openvino.dir/build.make:90: CMakeFiles/onnxruntime_providers_openvino.dir/home/junruizh/onnxruntime/onnxruntime/core/providers/openvino/backend_utils.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2159: CMakeFiles/onnxruntime_providers_openvino.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2964, in <module>
    sys.exit(main())
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2862, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 1721, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 868, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/junruizh/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/junruizh/onnxruntime/build/Android/RelWithDebInfo', '--config', 'RelWithDebInfo']' returned non-zero exit status 2.

[skottmckay]

The CI builds of OpenVINO we have must not have that warning available as find on a std::string returns std::size_t, which is unsigned and should always be a mismatch to int.

e.g. the Linux CI for OpenVINO does not have that warning available.

Your quickest way to test it out would be to disable the warning by commenting this out:

onnxruntime/cmake/CMakeLists.txt

Lines 750 to 752 in 8687011

	if (HAS_SHORTEN_64_TO_32)
	list(APPEND ORT_WARNING_FLAGS -Wshorten-64-to-32)
	endif()

[junruizh2021]

The CI builds of OpenVINO we have must not have that warning available as find on a std::string returns std::size_t, which is unsigned and should always be a mismatch to int.我们所拥有的 OpenVINO CI 版本一定不能有该警告，因为 std::string 上的find返回 std::size_t，它是无符号的，并且应该始终与int不匹配。

e.g. the Linux CI for OpenVINO does not have that warning available.例如，OpenVINO 的 Linux CI 没有该警告。

Your quickest way to test it out would be to disable the warning by commenting this out:测试它的最快方法是通过注释掉该警告来禁用警告：

onnxruntime/cmake/CMakeLists.txt

Lines 750 to 752 in 8687011

if (HAS_SHORTEN_64_TO_32)
list(APPEND ORT_WARNING_FLAGS -Wshorten-64-to-32)
endif()

Do you mean that this warning can be ignored?

[junruizh2021]

I encoutered another error in the process of building.

/home/junruizh/onnxruntime/onnxruntime/core/providers/openvino/backend_manager.cc:446:50: error: unused variable 'start_compute' [-Werror,-Wunused-variable]
  446 |   std::chrono::high_resolution_clock::time_point start_compute, end_compute;
      |                                                  ^~~~~~~~~~~~~
/home/junruizh/onnxruntime/onnxruntime/core/providers/openvino/backend_manager.cc:446:65: error: unused variable 'end_compute' [-Werror,-Wunused-variable]
  446 |   std::chrono::high_resolution_clock::time_point start_compute, end_compute;
      |                                                                 ^~~~~~~~~~~
2 errors generated.
gmake[2]: *** [CMakeFiles/onnxruntime_providers_openvino.dir/build.make:76: CMakeFiles/onnxruntime_providers_openvino.dir/home/junruizh/onnxruntime/onnxruntime/core/providers/openvino/backend_manager.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2159: CMakeFiles/onnxruntime_providers_openvino.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2964, in <module>
    sys.exit(main())
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2862, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 1721, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 868, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/junruizh/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/junruizh/onnxruntime/build/Android/RelWithDebInfo', '--config', 'RelWithDebInfo']' returned non-zero exit status 2.

[snnn]

@junruizh2021, these warnings were from OpenVino code that was contributed from Intel. You may help us contact your coworker to fix them, or you may add "--compile-no-warning-as-error" to your build command to suppress the warnings.

[snnn]

As Scott suggested, the line

int pos = output_name.find("/");

Should be changed to:

size_t pos = output_name.find("/");

[junruizh2021]

@snnn Thanks. I added the "--compile_no_warning_as_error" in the cmake command and it worked. But I encountered the "error: undefined symbol: Provider_GetHost". Did you know how to address this error?

[ 61%] Linking CXX shared module libonnxruntime_providers_openvino.so
ld.lld: error: undefined symbol: Provider_GetHost
>>> referenced by provider_ort_api_init.cc:27 (/home/junruizh/onnxruntime/onnxruntime/core/providers/shared_library/provider_ort_api_init.cc:27)
>>>               CMakeFiles/onnxruntime_providers_openvino.dir/home/junruizh/onnxruntime/onnxruntime/core/providers/shared_library/provider_ort_api_init.cc.o:(void std::__ndk1::__call_once_proxy[abi:ne180000]<std::__ndk1::tuple<onnxruntime::InitProviderOrtApi()::$_0&&>>(void*))
>>> referenced by provider_bridge_provider.cc:88 (/home/junruizh/onnxruntime/onnxruntime/core/providers/shared_library/provider_bridge_provider.cc:88)
>>>               CMakeFiles/onnxruntime_providers_openvino.dir/home/junruizh/onnxruntime/onnxruntime/core/providers/shared_library/provider_bridge_provider.cc.o:(_GLOBAL__sub_I_provider_bridge_provider.cc)
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/onnxruntime_providers_openvino.dir/build.make:431: libonnxruntime_providers_openvino.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2159: CMakeFiles/onnxruntime_providers_openvino.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2964, in <module>
    sys.exit(main())
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 2862, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 1721, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/junruizh/onnxruntime/tools/ci_build/build.py", line 868, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/junruizh/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/junruizh/onnxruntime/build/Android/RelWithDebInfo', '--config', 'RelWithDebInfo']' returned non-zero exit status 2.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[snnn]

PR #23567 would be helpful