[Build] Onnx runtime cross compilation with toolchain arm-linux-gnueabihf-9.1.0-gcc/g++ is failing due to 'from_chars' and 'std::enable_if' type error even with CXX 17

[work-dnd]

Describe the issue

When I am trying to build onnxruntime for arm by cross compiling on Linux using arm-linux-gnueabihf-9.1.0-gcc toolchain, I am getting the following errors -

1. /path/to/onnxruntime/include/onnxruntime/core/common/parse_string.h:38:41: error: no matching function for call to ‘from_chars(std::basic_string_view::const_pointer, std::basic_string_view::const_pointer, float&)’
   38 | const auto [ptr, ec] = std::from_chars(str.data(), str.data() + str.size(), parsed_value);

2. /path/to/toolchain/arm-linux-gnueabihf/include/c++/9.1.0/type_traits:2426:11: error: no type named ‘type’ in ‘struct std::enable_if<false, std::from_chars_result>’
   2426 | using enable_if_t = typename enable_if<_Cond, _Tp>::type;

and finally

make[2]: *** [CMakeFiles/onnxruntime_test_all.dir/build.make:188: /path/to/onnxruntime/onnxruntime/test/common/string_utils_test.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1700: CMakeFiles/onnxruntime_test_all.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
...
...
[ 74%] Linking CXX executable onnxruntime_shared_lib_test
[ 74%] Built target onnxruntime_shared_lib_test
make: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
File "/path/to/onnxruntime/tools/ci_build/build.py", line 3348, in
sys.exit(main())
File "/path/to/onnxruntime/tools/ci_build/build.py", line 3242, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/path/to/onnxruntime/tools/ci_build/build.py", line 2038, in build_targets
run_subprocess(cmd_args, env=env)
File "/path/to/onnxruntime/tools/ci_build/build.py", line 913, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "/path/to/onnxruntime/tools/python/util/run.py", line 50, in run
completed_process = subprocess.run(
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/path/to/onnxruntime/build/L

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
ONNX Runtime installed from (source or binary): source
ONNX Runtime version: 1.22.0
Python version: 3.10.16
GCC/Compiler: arm-linux-gnueabihf-9.1.0-gcc
cmake version: 3.30.

Summary generated during build:

-- Using custom protoc executable
Generated: /path/to/onnxruntime/build/Linux/Release/_deps/onnx-build/onnx/onnx-ml.proto
Generated: /path/to/onnxruntime/build/Linux/Release/_deps/onnx-build/onnx/onnx-operators-ml.proto
Generated: /path/to/onnxruntime/build/Linux/Release/_deps/onnx-build/onnx/onnx-data.proto

-- ******** Summary ********
-- CMake version : 3.30.8
-- CMake command : /usr/local/bin/cmake
-- System : Linux
-- C++ compiler : /opt/4K/gcc-sigmastar-9.1.0-2019.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-9.1.0-g++
-- C++ compiler version : 9.1.0
-- CXX flags : -std=c++17 -frtti -fexceptions -O0 -ldl -ffunction-sections -fdata-sections -Wno-error=attributes -Wnon-virtual-dtor
-- Build type : Release
-- Compile definitions : EIGEN_MPL2_ONLY;_GNU_SOURCE;__STDC_FORMAT_MACROS
-- CMAKE_PREFIX_PATH :
-- CMAKE_INSTALL_PREFIX : /usr/local
-- CMAKE_MODULE_PATH : /path/to/onnxruntime/cmake/external

-- ONNX version : 1.17.0
-- ONNX NAMESPACE : onnx
-- ONNX_USE_LITE_PROTO : ON
-- USE_PROTOBUF_SHARED_LIBS : OFF
-- Protobuf_USE_STATIC_LIBS : ON
-- ONNX_DISABLE_EXCEPTIONS : OFF
-- ONNX_DISABLE_STATIC_REGISTRATION : OFF
-- ONNX_WERROR : OFF
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_SHARED_LIBS :
-- BUILD_SHARED_LIBS : OFF

-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFF

-- Configured Eigen 3.4.90

-- Finished fetching external dependencies
NVCC_ERROR =
NVCC_OUT = no such file or directory
-- Configuring done (19.0s)
-- Generating done (12.3s)
-- Build files have been written to: /path/to/onnxruntime/build/Linux/Release
2025-03-24 15:53:08,482 build [INFO] - Building targets for Release configuration
2025-03-24 15:53:08,483 build [INFO] - /usr/local/bin/cmake --build /path/to/onnxruntime/build/Linux/Release --config Release -- -j4
[ 0%] Built target gen_onnx_proto
...
...

Urgency

The specific model I am trying to deploy on ARM platform requires Onnx runtime to be cross compiled with the above mentioned toolchain.

Target platform

arm

Build script

../build.sh
--config Release
--arm
--parallel
--cmake_extra_defines CMAKE_TOOLCHAIN_FILE=../arm-linux-custom.cmake
--skip_tests
--build_shared_lib

========================
arm-linux-custom.cmake

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

set(TOOLCHAIN_DIR /opt/4K/gcc-sigmastar-9.1.0-2019.11-x86_64_arm-linux-gnueabihf)
set(TOOLCHAIN_PREFIX ${TOOLCHAIN_DIR}/bin/arm-linux-gnueabihf-9.1.0)

set(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}-gcc)
set(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}-g++)
set(CMAKE_ASM_COMPILER ${TOOLCHAIN_PREFIX}-gcc)

set(CMAKE_FIND_ROOT_PATH ${TOOLCHAIN_DIR})

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

SET(CMAKE_LINKER_FLAGS "-shared")
SET(CMAKE_CXX_FLAGS "-std=c++17 -frtti -fexceptions -O0 -ldl")

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

if (${ARCH_ABI},armeabi-v7a)
add_definitions(-DARM_32BIT=1)
endif ()

if (${ARCH_ABI},arm64-v8a)
add_definitions(-DARM_64BIT=1)
endif ()

Error / output

1. /path/to/onnxruntime/include/onnxruntime/core/common/parse_string.h:38:41: error: no matching function for call to ‘from_chars(std::basic_string_view::const_pointer, std::basic_string_view::const_pointer, float&)’
   38 | const auto [ptr, ec] = std::from_chars(str.data(), str.data() + str.size(), parsed_value);

2. /path/to/toolchain/arm-linux-gnueabihf/include/c++/9.1.0/type_traits:2426:11: error: no type named ‘type’ in ‘struct std::enable_if<false, std::from_chars_result>’
   2426 | using enable_if_t = typename enable_if<_Cond, _Tp>::type;

and finally

make[2]: *** [CMakeFiles/onnxruntime_test_all.dir/build.make:188: /path/to/onnxruntime/onnxruntime/test/common/string_utils_test.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1700: CMakeFiles/onnxruntime_test_all.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
...
...
[ 74%] Linking CXX executable onnxruntime_shared_lib_test
[ 74%] Built target onnxruntime_shared_lib_test
make: *** [Makefile:146: all] Error 2
Traceback (most recent call last):
File "/path/to/onnxruntime/tools/ci_build/build.py", line 3348, in
sys.exit(main())
File "/path/to/onnxruntime/tools/ci_build/build.py", line 3242, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/path/to/onnxruntime/tools/ci_build/build.py", line 2038, in build_targets
run_subprocess(cmd_args, env=env)
File "/path/to/onnxruntime/tools/ci_build/build.py", line 913, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "/path/to/onnxruntime/tools/python/util/run.py", line 50, in run
completed_process = subprocess.run(
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', 'build-arm/Release', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

Visual Studio Version

No response

GCC / Compiler Version

No response

[snnn]

Ubuntu 18.04 is EOL. 20.04 is about EOL. So, please upgrade.

[edgchen1]

I think GCC 11.1 should have support for the float overload of from_chars in <charconv>.

https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html#status.iso.2017
See the "Elementary string conversions" feature.

We could add to the ifdef to check for GCC < 11.1, but I think we are just increasing the minimum required version (see #24148).

[BengtGustafsson]

It is impossible for us to upgrade beyond Ubuntu 20.04. You should not be this bleeding edge all the time. We have customers around the world and they are slow to update their OSes due to medtec regulatory issues. You can definitely not assume that everyone will upgrade before EOL of a version and should offer a grace period of at least 6 months after something goes out of support, preferrably longer.

I will push a PR that #ifs on the gcc version which we really require to be included in 1.21.1, which is due.... when? The release roadmap is not updated. Our release is due Q2 so I hope it is before end of May at least.

[snnn]

I have my difficulty.
First, I cannot use Ubuntu 20.04 at work after it is EOLed. So, in order to support such requests, I need to use my personal devices to build and test. I need to have macOS devices, RPi devices, qualcomm laptops that costs $$$ ... . I know Ubuntu is great, but this is beyond of what I can commit. I am still struggling to find the video cable for my RPi4.
Second, there is a conflict between compatibility and supporting new hardware features. For example, recently in the main branch we accepted a PR from ARM which added a dep to ONNX runtime for using SME2 instructions to speed up operator kernels. It requires a decent new GCC version to work. Because the toolchain has to be new enough to understand the new data types and CPU instructions. While ideally all such new features can be made optional, it requires a lot of resources. For example, we need to more pipelines to test with and without this feature. It is also difficulty for me to handle.
I am sorry.

[work-dnd]

I would like to update that after upgrading to Ubuntu 22.04, all the above mentioned errors did not occur and the runtime for arm cross-compiled successfully. I have also tested the same with the model inference on the target arm platform and it executed as per expectation.

Thanks @snnn for the quickest support.