Provided artifact name input during validation is empty

Process completed with exit code 1.

test_conv2d_device_perf subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/conv_perf --check-exit-code -a device_kernel_duration --op-support-count 1333 -t 5000 -m 'pytest tests/ttnn/perf_tests/operations/conv/test_conv2d_device_perf.py::test_run_conv2d_ops'"]' returned non-zero exit status 4.

test_run_conv2d_ops RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for pack_untilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/untilize.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/sliding_window/halo/device/kernels/compute/pack_untilize.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f1515898168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f15130f980c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f15131102b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f151311023e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f1512ea2def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f157c782ee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f1512ea2c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f151311008e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f15130b2d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f15130b1890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f15130b0d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f154e388253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f157c77dac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f157c80ea84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f1512fffed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f1512fdfb51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f1512fdf4cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f15131102b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f151311

Process completed with exit code 1.

test_perf_device_bare_metal_efficientnetb0[1-213.0] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/ttnn_efficientnetb0 --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/experimental/efficientnetb0/tests/pcc/test_ttnn_efficientnetb0.py'"]' returned non-zero exit status 4.

test_efficientnetb0_model[device_params0] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for transpose_wh_rm TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/eltwise_unary.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh_rm.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fd87c86b168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fd87a0cc80c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd87a0e32b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fd87a0e323e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fd879e75def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fd8e3768ee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fd879e75c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fd87a0e308e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fd87a085d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fd87a084890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fd87a083d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fd8b536b253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fd8e3763ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fd8e37f4a84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fd879fd2ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fd879fb2b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fd879fb24cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd87a0e32b8] --- /opt/venv/lib/python3.10/site-packages/ttn

Process completed with exit code 1.

test_perf_device_bare_metal_vovnet[1-159.0] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/ttnn_vovnet --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/experimental/vovnet/tests/pcc/test_tt_vovnet.py::test_vovnet_model_inference'"]' returned non-zero exit status 4.

test_vovnet_model_inference[device_params0-hf_hub:timm/ese_vovnet19b_dw.ra_in1k] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for conv_bmm_tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/conv/conv2d/device/kernels/conv_bmm_tilize.cpp:8, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f112e30d168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f112bb6e80c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f112bb852b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f112bb8523e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f112b917def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f11951fdee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f112b917c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f112bb8508e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f112bb27d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f112bb26890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f112bb25d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f1166e03253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f11951f8ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f1195289a84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f112ba74ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f112ba54b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f112ba544cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f112bb852b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/b

Process completed with exit code 1.

test_perf_device_bare_metal_swin_s[1-Swin_s-11.5] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/Swin_s --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/experimental/swin_s/tests/pcc/test_ttnn_swin_transformer.py::test_swin_s_transformer[pretrained_weight_true-0]'"]' returned non-zero exit status 4.

test_swin_s_transformer[pretrained_weight_true-0] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for transpose_wh_rm TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/eltwise_unary.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh_rm.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fc876015168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fc87387680c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fc87388d2b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fc87388d23e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fc87361fdef] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fc8dcf08ee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fc87361fc49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fc87388d08e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fc87382fd85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fc87382e890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fc87382dd78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fc8aeb06253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fc8dcf03ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fc8dcf94a84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fc87377ced8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fc87375cb51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fc87375c4cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fc87388d2b8] --- /opt/venv/lib/python3.10/site-pack

Process completed with exit code 1.

test_perf_device_bare_metal_swin_v2[1-Swin_v2-16.6] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/Swin_v2 --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/experimental/swin_v2/tests/pcc/test_ttnn_swin_v2_s.py::test_swin_s_transformer'"]' returned non-zero exit status 4.

test_swin_s_transformer[device_params0] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for transpose_xw_tiled TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/eltwise_unary.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/data_movement/permute/device/kernels/compute/transpose_xw_tiled.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f19c7e4f168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f19c56b080c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f19c56c72b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f19c56c723e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f19c5459def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f1a2ed68ee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f19c5459c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f19c56c708e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f19c5669d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f19c5668890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f19c5667d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f1a00953253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f1a2ed63ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f1a2edf4a84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f19c55b6ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f19c5596b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f19c55964cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f19c56c72b8] --- /opt/venv/lib/python3.10/site-packages/t

Process completed with exit code 1.

test_vanilla_unet_perf_device[1-206.0] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/unet_vanilla --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/demos/vision/segmentation/vanilla_unet/tests/test_unet_model.py::test_vanilla_unet_model'"]' returned non-zero exit status 4.

test_vanilla_unet_model[1-3-480-640-device_params0] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for convert_to_hwc TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/pack_untilize.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/experimental/cnn/convert_to_hwc/device/kernels/convert_to_hwc.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7ff39336e168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7ff390bcf80c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7ff390be62b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7ff390be623e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7ff390978def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ff3fa26eee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7ff390978c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7ff390be608e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7ff390b88d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7ff390b87890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7ff390b86d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ff3cbe7c253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ff3fa269ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ff3fa2faa84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7ff390ad5ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7ff390ab5b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7ff390ab54cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7ff390be62b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/buil

Process completed with exit code 1.

test_perf_device_mobilenetv2[10-3730] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/ttnn_mobilenetv2 --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/demos/vision/classification/mobilenetv2/tests/pcc/test_mobilenetv2.py'"]' returned non-zero exit status 4.

test_mobilenetv2[10-pretrained_weight_true-device_params0] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for conv_bmm_tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/conv/conv2d/device/kernels/conv_bmm_tilize.cpp:8, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fb589dfd168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fb58765e80c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fb5876752b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fb58767523e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fb587407def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fb5f0cf0ee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fb587407c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fb58767508e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fb587617d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fb587616890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fb587615d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fb5c28eb253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fb5f0cebac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fb5f0d7ca84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fb587564ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fb587544b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fb5875444cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fb5876752b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.s

Process completed with exit code 1.

test_mamba_perf_device[32-1.596] subprocess.CalledProcessError: Command '["python3 -m tracy -p -r -o /work/generated/profiler/ttnn_mamba --check-exit-code --op-support-count 1333 -t 5000 -m 'pytest models/demos/wormhole/mamba/tests/test_mamba_model.py::test_device_perf[1]'"]' returned non-zero exit status 4.

test_device_perf[1] RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception info: Failed to generate binaries for tilize_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception info: trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7, from /__w/tt-metal/tt-metal/docker-job/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize_wh.cpp:7, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21, from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13: /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory 10 | #include "vconst_verifier.h" | ^~~~~~~~~~~~~~~~~~~ compilation terminated. backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f91487fd168] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f914605e80c] --- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f91460752b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f914607523e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9145e07def] --- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f91af6eaee8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9145e07c49] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f914607508e] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9146017d85] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9146016890] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9146015d78] --- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f91812eb253] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f91af6e5ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f91af776a84] backtrace: --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9145f64ed8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9145f44b51] --- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&) --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9145f444cb] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f91460752b8] --- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f914607523e] --

Benchmark data directory 'generated/benchmark_data' does not exist

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.

Field "model_name" in OpTest has conflict with protected namespace "model_". You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

Support for class-based `config` is deprecated, use ConfigDict instead.