[onert] Investigate why focal complains with kMaxSmallSize=6

[glistening]

After #15002, build breaks on focal.

In member function 'void nnfw::cker::Einsum::contractOperands(std::vector<nnfw::cker::Tensor>&, std::vector<bool>&, nnfw::cker::Tensor*)':

cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]

In file included from /home/jenkins/jenkins_agent/workspace/nnfw/master/push-nnpackage-test/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:

/home/jenkins/jenkins_agent/workspace/nnfw/master/push-nnpackage-test/runtime/compute/cker/include/cker/operation/Einsum.h:874:11: note: 'output_shape' declared here

Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());
      |           ^~~~~~~~~~~~

Though it disappeared with #15101, I am not sure why it happens and why it disappeared.

There are several possibilities:

• (1) focal (old toolchain) build error was false alarm.
• (2) focal error is valid, it was fixed with [cker] Fix array-bounds build error: Refactor Shape.h to use C++17 std::variant #15101.
• (3) focal error is valid, it was not fixed yet, but just crept in our code.

I don't think it is in hurry, and don't want to put much time on this case.

Note

1. focal (Ubuntu 20.04) will reach the end of support on May 31, 2025.
2. Our CI does not check the build on focal. (Maybe due to 1.)
3. On the recent gcc, the error does not occurs.

However, for the little possibility of (3) and my curiosity,
it may be good to investigate on this if time permits.

• Set up the focal build environment
• Check einsum requires changes as kMaxSmallSize becomes 6.
• Check 32-bit alignment in union

  union {
    int32_t _dims[kMaxSmallSize];
    int32_t *_dims_pointer{nullptr};
  };

[glistening]

Set up the focal build environment

In short, I cannot reproduce the focal build error.

I tried to build on focal in weekend, on my home ubuntu (focal).
There was no problem with #15002 and without #15101.
It is the same with focal docker image, whici I had used before.

focal/Dockerfile installs gcc via build-essential. It should be gcc 9.4.0.

I cannot reproduce the error that our daily ( ? ) CI showed.

I will try more after @hseok-oh returns.

Add, 4/14 11:44 I confirmed from https://github.sec.samsung.net/one-project/ONE/actions/runs/16611144/job/52210645

./nnfw configure \
	-DCMAKE_BUILD_TYPE=release \
	-DNNFW_OVERLAY_DIR=/__w/ONE/ONE/Product/x86_64-linux.release/overlay \
	-DEXTERNALS_BUILD_THREADS=96 \
	-DTRIX_ENGINE_PREFIX=/opt/trinity -DENABLE_COVERAGE=OFF
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0

Add, 4/14 11:44 The error occurs in release build.

[ 94%] Building CXX object runtime/tests/tools/tflite_run/CMakeFiles/tflite_run.dir/src/tflite_run.cc.o
[ 94%] Building CXX object runtime/tests/tools/tflite_run/CMakeFiles/tflite_run.dir/src/args.cc.o
In member function 'void nnfw::cker::Einsum::contractOperands(std::vector<nnfw::cker::Tensor>&, std::vector<bool>&, nnfw::cker::Tensor*)':
cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]
In file included from /__w/ONE/ONE/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:
/__w/ONE/ONE/runtime/compute/cker/include/cker/operation/Einsum.h:874:11: note: 'output_shape' declared here
  874 |     Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());
      |           ^~~~~~~~~~~~
[ 94%] Building CXX object runtime/tests/tools/onert_run/CMakeFiles/onert_run.dir/src/args.cc.o
[ 94%] Building CXX object runtime/tests/tools/tflite_run/CMakeFiles/tflite_run.dir/src/tensor_dumper.cc.o
[ 94%] Building CXX object runtime/tests/tools/onert_train/CMakeFiles/onert_train.dir/src/args.cc.o
cc1plus: all warnings being treated as errors
make[3]: *** [runtime/onert/backend/cpu/CMakeFiles/onert_backend_cpu.dir/build.make:297: runtime/onert/backend/cpu/CMakeFiles/onert_backend_cpu.dir/ops/EinsumLayer.cc.o] Error 1
make[3]: *** Waiting for unfinished jobs....

[ragmani]

You can reproduce the focal build error on rs7-runtimentools-docker-local.bart.sec.samsung.net/one-devtools:focal-proxy container.

[ragmani]

I tried to build 068c991 on the container again. The same error still occurs.

In member function 'void nnfw::cker::Einsum::contractOperands(std::vector<nnfw::cker::Tensor>&, std::vector<bool>&, nnfw::cker::Tensor*)':
cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]
In file included from /home/jang/git/ragmani/onert/ONE/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:
/home/jang/git/ragmani/onert/ONE/runtime/compute/cker/include/cker/operation/Einsum.h:874:11: note: 'output_shape' declared here
  874 |     Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());
      |           ^~~~~~~~~~~~
[ 83%] Building CXX object externals/tflite-2.16.1/CMakeFiles/tensorflow-lite-2.16.1.dir/home/jang/git/ragmani/onert/ONE/externals/TENSORFLOW-2.16.1/tensorflow/lite/simple_memory_arena.cc.o
cc1plus: all warnings being treated as errors
gmake[3]: *** [runtime/onert/backend/cpu/CMakeFiles/onert_backend_cpu.dir/build.make:297: runtime/onert/backend/cpu/CMakeFiles/onert_backend_cpu.dir/ops/EinsumLayer.cc.o] Error 1
gmake[3]: *** Waiting for unfinished jobs....

However, its parent fcf86b4 builds successfully on the container.

[glistening]

@ragmani I read your comment just after I've also succeeded to reproduce with release build.

[ 80%] Built target nnfw_api_pybind
[ 80%] Building CXX object runtime/onert/core/CMakeFiles/test_onert_core.dir/src/util/ObjectManager.test.cc.o
In member function 'void nnfw::cker::Einsum::contractOperands(std::vector<nnfw::cker::Tensor>&, std::vector<bool>&, nnfw::cker::Tensor*)':
cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]
In file included from /mnt/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:
/mnt/runtime/compute/cker/include/cker/operation/Einsum.h:874:11: note: 'output_shape' declared here
  874 |     Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());

[glistening]

In short, it seems the bug of gcc 9.4 optimization.

It does not occur in debug build.
Just turning off optimization for Einsum::contractOperands removes the error.

diff --git a/runtime/compute/cker/include/cker/operation/Einsum.h b/runtime/compute/cker/include/cker/operation/Einsum.h
index aaed0e505a..3915162cd8 100644
--- a/runtime/compute/cker/include/cker/operation/Einsum.h
+++ b/runtime/compute/cker/include/cker/operation/Einsum.h
@@ -854,6 +854,7 @@ private:
   // when the matrix shape is [1,1]; in this case BatchMatMul functor would be
   // very inefficient. The functor should detect if this is the case and perform
   // componentwise multiplication functor instead.
+  __attribute__((optimize("O0")))
   void contractOperands(std::vector<Tensor> &inputs, std::vector<bool> &swap_free_and_contract,
                         Tensor *output)
   {

I confirmed __builtin_memcpy is genereated during inlining for Einsum by dumping generated assembly code.

[glistening]

For recording,

In file included from /__w/ONE/ONE/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:
/__w/ONE/ONE/runtime/compute/cker/include/cker/operation/Einsum.h:874:11: note: 'output_shape' declared here
  874 |     Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());

1.

gcc 9.4.0 with -O3 considers the argument as constant.
It is the reason why it occurs on build time.

gcc thinks it is constant at least within the compilation unit of EinsumLayer.cc.

You can reproduce the error simply replacing

Shape output_shape(old_output_shape.DimensionsCount() + inputs.size());

with

Shape output_shape(6);    // or (4) also shows the same error

cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]
In file included from /mnt/runtime/onert/backend/cpu/ops/EinsumLayer.cc:19:
/mnt/runtime/compute/cker/include/cker/operation/Einsum.h:875:11: note: 'output_shape' declared here
  875 |     Shape output_shape(6);
      |           ^~~~~~~~~~~~

Shape output_shape(8);

No error occurs.

2. sizeof(Shape)

sizeof(Shape) = 32 for kMaxSmallSize = 6
sizeof(Shape) = 24 for kMaxSmallSize = 4

Basically, on x64, it is aligned as 64 bit.

clss Shape {
  int32_t _size;
  union {
    int32_t _dims[kMaxSmallSize];
    int32_t *_dims_pointer{nullptr};
  };

_size is 4 byte, but aligned in 8 bytes. It occuppies 8 bytes.
union 6 x 4 = 24 (kMaxSmallSize = 6), 8 + 24 = 32
union 4 x 4 = 16 (kMaxSmallSize = 4), 4 + 24 = 28

3.

In member function 'void nnfw::cker::Einsum::contractOperands(std::vector<nnfw::cker::Tensor>&, std::vector<bool>&, nnfw::cker::Tensor*)':
cc1plus: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset [33, 36] is out of the bounds [0, 32] of object 'output_shape' with type 'nnfw::cker::Shape' [-Werror=array-bounds]

Shape output_shape(6);
for (int i = 0; i < old_output_shape.DimensionsCount(); i++)
{
    output_shape.SetDim(i, old_output_shape.Dims(i));
}

for (size_t i = 0; i < inputs.size(); ++i)
{
    const int32_t free_axis =
    inputs[i].shape.DimensionsCount() - (swap_free_and_contract[i] ? 1 : 2);
    output_shape.SetDim(i + old_output_shape.DimensionsCount(), inputs[i].shape.Dims(free_axis));
}
bool adj_x = swap_free_and_contract[0];
bool adj_y = !swap_free_and_contract[1];

// allocateTemp(output_shape, output);

Commenting out allocateTemp removes the build error.

4.

void allocateTemp(const Shape &shape, Tensor *output)
  {
    output->shape.ReplaceWith(shape.DimensionsCount(), shape.DimsData());
    temp_operand.emplace_back(std::make_unique<float[]>(shape.FlatSize()));
    output->buffer = temp_operand.back().get();
  }

seems to be inlined.

Note: The following code is the generated code with allocateTemp.

.LLSDACSE14716:
	.section	.text._ZN4nnfw4cker6Einsum16contractOperandsERSt6vectorINS0_6TensorESaIS3_EERS2_IbSaIbEEPS3_,"axG",@progbits,_ZN4nnfw4cker6Einsum16contractOperandsERSt6vectorINS0_6TensorESaIS3_EERS2_IbSaIbEEPS3_,comdat
	.size	_ZN4nnfw4cker6Einsum16contractOperandsERSt6vectorINS0_6TensorESaIS3_EERS2_IbSaIbEEPS3_, .-_ZN4nnfw4cker6Einsum16contractOperandsERSt6vectorINS0_6TensorESaIS3_EERS2_IbSaIbEEPS3_
	.section	.text._ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEliNS0_5__ops15_Iter_comp_iterIZN4nnfw4cker6Einsum13reduceOperandIfEEvRKNSA_11InputTensorIT_EERKS3_INSA_13DimensionTypeESaISI_EERKS5_PS5_SP_PbPNSA_6TensorEEUliiE_EEEvSE_T0_SV_T1_T2_,"axG",@progbits,_ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEliNS0_5__ops15_Iter_comp_iterIZN4nnfw4cker6Einsum13reduceOperandIfEEvRKNSA_11InputTensorIT_EERKS3_INSA_13DimensionTypeESaISI_EERKS5_PS5_SP_PbPNSA_6TensorEEUliiE_EEEvSE_T0_SV_T1_T2_,comdat
	.p2align 4
	.weak	_ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEliNS0_5__ops15_Iter_comp_iterIZN4nnfw4cker6Einsum13reduceOperandIfEEvRKNSA_11InputTensorIT_EERKS3_INSA_13DimensionTypeESaISI_EERKS5_PS5_SP_PbPNSA_6TensorEEUliiE_EEEvSE_T0_SV_T1_T2_
	.type	_ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEliNS0_5__ops15_Iter_comp_iterIZN4nnfw4cker6Einsum13reduceOperandIfEEvRKNSA_11InputTensorIT_EERKS3_INSA_13DimensionTypeESaISI_EERKS5_PS5_SP_PbPNSA_6TensorEEUliiE_EEEvSE_T0_SV_T1_T2_, @function
_ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPiSt6vectorIiSaIiEEEEliNS0_5__ops15_Iter_comp_iterIZN4nnfw4cker6Einsum13reduceOperandIfEEvRKNSA_11InputTensorIT_EERKS3_INSA_13DimensionTypeESaISI_EERKS5_PS5_SP_PbPNSA_6TensorEEUliiE_EEEvSE_T0_SV_T1_T2_:
.LFB20893:
	.cfi_startproc
 << very complex ... including __bulitin_memcpy_chk >>

I guess something went wrong inlining allocateTemp with kSmallMaxSize = 6.

The error is gone when we don't permit the aggressive optimization for contractOperands.

Now, I am almost sure that it is a bug of gcc 9.4 with -O3.

I would choose to fix the focal gcc build error with __attribute__((optimize("O0"))),
then, to remove the line after focal is out of support in our repo.

I would like to hear the opinion about contractOperands from @hseok-oh (contractOperands writer).

However, I was pushed last Friday that build break on focal is in urgent. (though I don't agree.) The PR is merged and 1 additional fix PR is merged.

I hope the merged PR fixed the bug of gcc focal. (It is likely since gcc 9.4.0 does not try to optimize aggressively which has c++17 and std::array and std::vector like C-style code.) and new code dose not have side effects.