Nightly tt-metal L2 tests #3455
fvranicTTtt-metal-l2-nightly.yaml
on: workflow_dispatch
build-artifact
/
download-artifacts
generate-arch-matrix
3s
build-artifact
/
parse-platform
5s
build-artifact
/
...
/
check-docker-images
15s
build-artifact
/
...
/
🐳️ Build Ubuntu images
build-artifact
/
...
/
🐳️ Build ManyLinux image
build-artifact
/
...
/
🔄 Update latest tag
build-artifact
/
🛠️ Build Release ubuntu 22.04
5m 34s
build-artifact
/
...
/
🐍 Build wheel (Python 3.10)
5m 31s
Matrix: cpp-unit-tests
Matrix: didt-tests
Waiting for pending jobs
Matrix: fast-dispatch-unit-tests
Matrix: metal-iommu-unit-tests
Matrix: models-unit-tests
Matrix: run-profiler-regression
Matrix: sd-unit-tests
Matrix: test-ttnn-tutorials
Waiting for pending jobs
Matrix: tt-cnn-unit-tests
Matrix: tt-metal-l2-tests
Matrix: tt-train-cpp-unit-tests
Matrix: ttnn-ops-docs-check
Waiting for pending jobs
Annotations
142 errors and 229 warnings
|
sd-unit-tests (blackhole, P100) / blackhole P100 debug_tools and device and dispatch
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_rotate_half.py#L22
test_rotate_half[shape0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_hw TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_hw.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f97fa5cb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f97f7e2c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f97f7e432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f97f7e4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f97f7bd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f982af17ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f97f7bd5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f97f7e4308e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f97f7e432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f97f7e4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f97f7bd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f982af17ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f97f7bd5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f97f7e4308e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 llk
Process completed with exit code 139.
|
|
sd-unit-tests (blackhole, P100) / blackhole P100 llk
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_transpose.py#L804
test_transpose_4d_wh_tile[shape1]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fafbc413168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fafb9c7480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fafb9c8b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fafb9c8b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fafb9a1ddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fb028cdcee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fafb9a1dc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fafb9c8b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fafb9c2dd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fafb9c2c890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fafb9c2bd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fb026393253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fb028cd7ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fb028d68a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fafb9b7aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fafb9b5ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fafb9b5a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fafb9c8b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fafb9c8b23e]
--- /opt/venv/l
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_transpose.py#L804
test_transpose_4d_wh_tile[shape1]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f881ffc7168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f881d82880c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f881d83f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f881d83f23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f881d5d1def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f885a886ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f881d5d1c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f881d83f08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f881d7e1d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f881d7e0890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f881d7dfd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f8857f86253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f885a881ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f885a912a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f881d72eed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f881d70eb51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f881d70e4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f881d83f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f881d83f23e]
--- /opt/venv/l
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_padding_test.py#L131
test_run_padding_and_add_test[input_tensor_shape0-output_tensor_shape0-input_tensor_start0-0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_binary.h:7,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f4ab87cb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f4ab602c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4ab60432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4ab604323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f4ab5dd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f4ae90feee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f4ab5dd5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f4ab604308e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4ab60432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4ab604323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f4ab5dd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f4ae90feee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f4ab5dd5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.s
|
|
fast-dispatch-unit-tests (blackhole, P150) / sweep blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / sweep blackhole P150:
tests/tt_eager/python_api_testing/sweep_tests/pytests/helper_funcs/test_linear.py#L27
test_linear_no_bias[input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f8d21ce7168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f8d1f54880c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f8d1f55f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f8d1f55f23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f8d1f2f1def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f8d5c64aee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f8d1f2f1c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f8d1f55f08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f8d1f501d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f8d1f500890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f8d1f4ffd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f8d59d76253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f8d5c645ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f8d5c6d6a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f8d1f44eed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f8d1f42eb51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f8d1f42e4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f8d1f55f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f8d1f55f23e]
--- /opt/venv/
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_matmul_dram_sharded.py#L254
test_matmul_in1_dram_sharded_with_program_cache[False-True-True-32-8192-1280-None-grid_size0-DataType.BFLOAT16-DataType.BFLOAT8_B-DataType.BFLOAT16-no_bias-no_packer_l1_acc-HiFi2]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f195f3cb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f195cc2c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f195cc432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f195cc4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f195c9d5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f198fc9cee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f195c9d5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f195cc4308e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f195cc432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f195cc4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f195c9d5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f198fc9cee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so
|
|
cpp-unit-tests (blackhole, P150b) / Nightly TTNN university lab examples blackhole P150b
Process completed with exit code 134.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_create_qkv_heads.py#L327
test_nlp_create_q_and_kv_heads_separate_test[2-4096-96-8-8-64-2-8-True-BFLOAT8_B]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh_sharded TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/transformer/split_query_key_value_and_split_heads/device/kernels/compute/transpose_wh_sharded.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f83a16eb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f839ef4c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f839ef632b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f839ef6323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f839ecf5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f83d205fee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f839ecf5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f839ef6308e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f839ef632b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f839ef6323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f839ecf5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f83d205fee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f839ecf5c49]
--- /opt/venv/lib/python3.10/site-packages/ttn
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 debug_tools and device and dispatch
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_bcast.py#L134
test_bcast[ShardOrientation.COL_MAJOR-1-1-BcastOpMath.MUL-DataType.BFLOAT16-DataType.BFLOAT8_B-512-1280-40-shard_grid6-ShardStrategy.BLOCK]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_h_sharded_optimised TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_h_sharded_optimised.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fcf5e9bf168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fcf5c22080c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fcf5c2372b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fcf5c23723e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fcf5bfc9def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fcfad3f9ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fcf5bfc9c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fcf5c23708e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fcf5c1d9d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fcf5c1d8890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fcf5c1d7d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fcfaaaf9253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fcfad3f4ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fcfad485a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fcf5c126ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fcf5c106b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fcf5c1064cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fcf5c2372b
|
|
cpp-unit-tests (blackhole, P150b) / All C++ blackhole P150b:
tests/tt_metal/tt_metal/test_sdpa_reduce_c.cpp#L296
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for compute TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc0 build failed. Log: In file included from /work/tests/tt_metal/tt_metal/test_kernels/misc/sdpa/reduce_c/compute.cpp:8,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:42,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void sub_exp_block_bcast_cols_inplace(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:322:66: error: division by zero [-Werror=div-by-zero]
322 | uint32_t granularity = (cols >= SUB_EXP_GRANULARITY) ? (cols / SUB_EXP_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:430:76: error: division by zero [-Werror=div-by-zero]
430 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols_inplace(uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:482:72: error: division by zero [-Werror=div-by-zero]
482 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_scalar_inplace(uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:520:48: error: division by zero [-Werror=div-by-zero]
520 | constexpr uint32_t granularity = num_tiles / STATS_GRANULARITY;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void matmul_reduce(uint32_t, const uint32_t&)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:1274:46: error: division by zero [-Werror=div-by-zero]
1274 | constexpr uint32_t in0_num_subblocks = M / STATS_GRANULARITY;
cc1plus: all warnings being treated as errors
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7ff434cf82d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7ff434f82b2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ff434f99468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ff434f993ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ff434d28f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ff433d59ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ff434d28d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ff434f9923e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ff434f3c095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ff434f3aba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ff434f3a088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ff4340ce253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ff433d54ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ff433de5a84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7ff434e89318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7ff434e684b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7ff434e67e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ff434f99468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ff434f993ee]
--- /work/build/
|
|
cpp-unit-tests (blackhole, P150b) / All C++ blackhole P150b
Process completed with exit code 1.
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 llk
Process completed with exit code 139.
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 All C++, API and user kernel path tests
Process completed with exit code 1.
|
|
sd-unit-tests (blackhole, P100) / blackhole P100 All C++, API and user kernel path tests
Process completed with exit code 1.
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 debug_tools and device and dispatch
Process completed with exit code 139.
|
|
cpp-unit-tests (wormhole_b0, N300) / distributed wormhole_b0 N300
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_rotary_embedding_llama.py#L493
test_rotary_embedding_llama[0.9997-DataType.BFLOAT16-11-3-128-decode_8]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fef534e3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fef50d4480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fef50d5b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fef50d5b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fef50aeddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fef97d94ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fef50aedc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fef50d5b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fef50cfdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fef50cfc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fef50cfbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fef954c5253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fef97d8fac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fef97e20a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fef50c4aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fef50c2ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fef50c2a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fef50d5b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0
|
|
cpp-unit-tests (blackhole, P100a) / dispatch multicmd queue blackhole P100a:
tests/tt_metal/tt_metal/dispatch/dispatch_trace/test_EnqueueTrace.cpp#L209
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_sfpu TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /work/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /work/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /work/tt_metal/hw/inc/api/compute/common.h:7,
from /work/tt_metal/kernels/compute/eltwise_sfpu.cpp:6,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7ffba7b512d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7ffba7ddbb2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7ffba7ce2318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7ffba7cc14b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7ffba7cc0e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
" thrown in the test body.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch multicmd queue blackhole P100a:
tests/tt_metal/tt_metal/dispatch/dispatch_trace/test_EnqueueTrace.cpp#L150
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_sfpu TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /work/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /work/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /work/tt_metal/hw/inc/api/compute/common.h:7,
from /work/tt_metal/kernels/compute/eltwise_sfpu.cpp:6,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7ffba7b512d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7ffba7ddbb2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7ffba7ce2318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7ffba7cc14b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7ffba7cc0e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
" thrown in the test body.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch multicmd queue blackhole P100a:
tests/tt_metal/tt_metal/dispatch/dispatch_trace/test_EnqueueTrace.cpp#L99
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_sfpu TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /work/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /work/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /work/tt_metal/hw/inc/api/compute/common.h:7,
from /work/tt_metal/kernels/compute/eltwise_sfpu.cpp:6,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7ffba7b512d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7ffba7ddbb2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7ffba7ce2318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7ffba7cc14b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7ffba7cc0e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
" thrown in the test body.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch multicmd queue blackhole P100a:
tests/tt_metal/tt_metal/dispatch/dispatch_program/test_EnqueueProgram.cpp#L1847
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for read_my_coordinates TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /work/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /work/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /work/tt_metal/hw/inc/api/compute/common.h:7,
from /work/tests/tt_metal/tt_metal/test_kernels/misc/read_my_coordinates.cpp:16,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7ffba7b512d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7ffba7ddbb2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7ffba7ce2318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7ffba7cc14b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7ffba7cc0e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7ffba7df2468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7ffba7df23ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7ffba7b81f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ffba6bb2ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7ffba7b81d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7ffba7df223e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7ffba7d95095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7ffba7d93ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7ffba7d93088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ffba6f27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ffba6badac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ffba6c3ea84]
" thrown in the test body.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch multicmd queue blackhole P100a
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_create_qkv_heads.py#L327
test_nlp_create_q_and_kv_heads_separate_test[2-4096-96-8-8-64-2-8-True-BFLOAT8_B]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh_sharded TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/transformer/split_query_key_value_and_split_heads/device/kernels/compute/transpose_wh_sharded.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x746ed20d2168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x746ecf93380c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x746ecf94a2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x746ecf94a23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x746ecf6dcdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x746f40349ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x746ecf6dcc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x746ecf94a08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x746ecf8ecd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x746ecf8eb890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x746ecf8ead78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x746f3d87a253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x746f40344ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x746f403d5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x746ecf839ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x746ecf819b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x746ecf8194cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x746ecf94a2b8]
--- /opt/venv/
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn builder unit tests wormhole_b0 N150
Process completed with exit code 139.
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn builder unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_builder.py#L168
test_conv2d[sharding_strategy0-kernel_config0-1-channel_config0-input_size1-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for conv_bmm_tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/conv/conv2d/device/kernels/conv_bmm_tilize.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f4e51ae3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f4e4f34480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e4f35b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4e4f35b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f4e4f0eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f4e96404ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f4e4f0edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f4e4f35b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f4e4f2fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f4e4f2fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f4e4f2fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f4e93b7e253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f4e963ffac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f4e96490a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f4e4f24aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f4e4f22ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f4e4f22a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e4f35b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_me
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn builder unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_builder.py#L168
test_conv2d[sharding_strategy0-kernel_config0-1-channel_config0-input_size0-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for conv_bmm_tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/conv/conv2d/device/kernels/conv_bmm_tilize.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f4e51ae3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f4e4f34480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e4f35b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4e4f35b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f4e4f0eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f4e96404ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f4e4f0edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f4e4f35b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f4e4f2fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f4e4f2fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f4e4f2fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f4e93b7e253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f4e963ffac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f4e96490a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f4e4f24aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f4e4f22ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f4e4f22a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e4f35b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_me
|
|
cpp-unit-tests (blackhole, P100a) / All C++ blackhole P100a:
tests/tt_metal/tt_metal/test_sdpa_reduce_c.cpp#L296
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for compute TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc0 build failed. Log: In file included from /work/tests/tt_metal/tt_metal/test_kernels/misc/sdpa/reduce_c/compute.cpp:8,
from /work/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:42,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void sub_exp_block_bcast_cols_inplace(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:322:66: error: division by zero [-Werror=div-by-zero]
322 | uint32_t granularity = (cols >= SUB_EXP_GRANULARITY) ? (cols / SUB_EXP_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:430:76: error: division by zero [-Werror=div-by-zero]
430 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols_inplace(uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:482:72: error: division by zero [-Werror=div-by-zero]
482 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_scalar_inplace(uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:520:48: error: division by zero [-Werror=div-by-zero]
520 | constexpr uint32_t granularity = num_tiles / STATS_GRANULARITY;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void matmul_reduce(uint32_t, const uint32_t&)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:1274:46: error: division by zero [-Werror=div-by-zero]
1274 | constexpr uint32_t in0_num_subblocks = M / STATS_GRANULARITY;
cc1plus: all warnings being treated as errors
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7f35d65b02d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7f35d683ab2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7f35d6851468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7f35d68513ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7f35d65e0f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f35d5611ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7f35d65e0d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7f35d685123e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7f35d67f4095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7f35d67f2ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7f35d67f2088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f35d5986253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f35d560cac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f35d569da84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7f35d6741318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7f35d67204b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7f35d671fe26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7f35d6851468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7f35d68513ee]
--- /work/build/
|
|
cpp-unit-tests (blackhole, P100a) / All C++ blackhole P100a
Process completed with exit code 1.
|
|
cpp-unit-tests (blackhole, P150b) / dispatch blackhole P150b
Process completed with exit code 134.
|
|
cpp-unit-tests (wormhole_b0, N300) / All C++ wormhole_b0 N300:
tests/tt_metal/tt_metal/test_sdpa_reduce_c.cpp#L296
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for compute TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc0 build failed. Log: In file included from /work/tests/tt_metal/tt_metal/test_kernels/misc/sdpa/reduce_c/compute.cpp:8,
from /work/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:37,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void sub_exp_block_bcast_cols_inplace(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:322:66: error: division by zero [-Werror=div-by-zero]
322 | uint32_t granularity = (cols >= SUB_EXP_GRANULARITY) ? (cols / SUB_EXP_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:430:76: error: division by zero [-Werror=div-by-zero]
430 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols_inplace(uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:482:72: error: division by zero [-Werror=div-by-zero]
482 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_scalar_inplace(uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:520:48: error: division by zero [-Werror=div-by-zero]
520 | constexpr uint32_t granularity = num_tiles / STATS_GRANULARITY;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void matmul_reduce(uint32_t, const uint32_t&)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:1274:46: error: division by zero [-Werror=div-by-zero]
1274 | constexpr uint32_t in0_num_subblocks = M / STATS_GRANULARITY;
cc1plus: all warnings being treated as errors
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7f3bdc0062d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7f3bdc290b2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7f3bdc2a7468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7f3bdc2a73ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7f3bdc036f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f3bdb067ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7f3bdc036d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7f3bdc2a723e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7f3bdc24a095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7f3bdc248ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7f3bdc248088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f3bdb3dc253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f3bdb062ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f3bdb0f3a84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7f3bdc197318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7f3bdc1764b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7f3bdc175e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7f3bdc2a7468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7f3bdc2a73ee]
--- /work/buil
|
|
cpp-unit-tests (wormhole_b0, N300) / All C++ wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_matmul_1d_gather_in0.py#L601
test_multi_core_matmul_1d_wh[3-True-None-3-32-2304-3840-DataType.BFLOAT16-DataType.BFLOAT8_B-MathFidelity.HiFi2-True-False-grid4-no_bias]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation_gathered TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation_gathered.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f0bd3bed168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f0bd144e80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f0bd14652b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f0bd146523e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f0bd11f7def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f0c18498ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f0bd11f7c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f0bd146508e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f0bd1407d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f0bd1406890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f0bd1405d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f0c15bc9253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f0c18493ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f0c18524a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f0bd1354ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f0bd1334b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f0bd13344cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/bu
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 All C++, API and user kernel path tests
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_transpose.py#L878
test_transpose_former_failures[memory_config0-config7]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_xw_rm_single_tile_size TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/eltwise_unary.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/permute/device/kernels/compute/transpose_xw_rm_single_tile_size.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fa90bae3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fa90934480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fa90935b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fa90935b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fa9090eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fa9503f6ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fa9090edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fa90935b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fa9092fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fa9092fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fa9092fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fa94db27253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fa9503f1ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fa950482a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fa90924aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fa90922ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fa90922a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fa90935b2b8]
--- /opt/venv/lib/python3.10/
|
|
cpp-unit-tests (blackhole, P150b) / dispatch multicmd queue blackhole P150b
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_transpose.py#L878
test_transpose_former_failures[memory_config0-config7]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_xw_rm_single_tile_size TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/eltwise_unary.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/permute/device/kernels/compute/transpose_xw_rm_single_tile_size.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f85e7896168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f85e50f780c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f85e510e2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f85e510e23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f85e4ea0def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f862c03bee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f85e4ea0c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f85e510e08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f85e50b0d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f85e50af890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f85e50aed78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f862976c253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f862c036ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f862c0c7a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f85e4ffded8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f85e4fddb51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f85e4fdd4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f85e510e2b8]
--- /opt/venv/lib/python3.10/
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / sweep wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / sweep wormhole_b0 N300:
tests/tt_eager/python_api_testing/sweep_tests/pytests/helper_funcs/test_linear.py#L27
test_linear_no_bias[input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f2bdc401168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f2bd9c6280c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f2bd9c792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f2bd9c7923e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f2bd9a0bdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f2c20cbdee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f2bd9a0bc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f2bd9c7908e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f2bd9c1bd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f2bd9c1a890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f2bd9c19d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f2c1e46f253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f2c20cb8ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f2c20d49a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f2bd9b68ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f2bd9b48b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f2bd9b484cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f2bd9c792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f2bd9c7923e]
|
|
cpp-unit-tests (blackhole, P150b) / distributed blackhole P150b
Process completed with exit code 134.
|
|
cpp-unit-tests (wormhole_b0, N300) / Nightly TTNN university lab examples wormhole_b0 N300
Process completed with exit code 134.
|
|
cpp-unit-tests (wormhole_b0, N150) / All C++ wormhole_b0 N150:
tests/tt_metal/tt_metal/test_sdpa_reduce_c.cpp#L296
unknown file
C++ exception with description "TT_THROW @ /work/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for compute TT_THROW @ /work/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc0 build failed. Log: In file included from /work/tests/tt_metal/tt_metal/test_kernels/misc/sdpa/reduce_c/compute.cpp:8,
from /work/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:37,
from /work/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void sub_exp_block_bcast_cols_inplace(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:322:66: error: division by zero [-Werror=div-by-zero]
322 | uint32_t granularity = (cols >= SUB_EXP_GRANULARITY) ? (cols / SUB_EXP_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols(uint32_t, uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:430:76: error: division by zero [-Werror=div-by-zero]
430 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_cols_inplace(uint32_t, uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:482:72: error: division by zero [-Werror=div-by-zero]
482 | constexpr uint32_t granularity = (cols >= DHT_GRANULARITY) ? (cols / DHT_GRANULARITY) : 1;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void mul_block_bcast_scalar_inplace(uint32_t)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:520:48: error: division by zero [-Werror=div-by-zero]
520 | constexpr uint32_t granularity = num_tiles / STATS_GRANULARITY;
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp: In function 'void matmul_reduce(uint32_t, const uint32_t&)':
/work/ttnn/cpp/ttnn/operations/transformer/sdpa/device/kernels/compute/compute_common.hpp:1274:46: error: division by zero [-Werror=div-by-zero]
1274 | constexpr uint32_t in0_num_subblocks = M / STATS_GRANULARITY;
cc1plus: all warnings being treated as errors
backtrace:
--- /work/build/lib/libtt_metal.so(+0x54e2d8) [0x7fd4e78f72d8]
--- /work/build/lib/libtt_metal.so(+0x7d8b2c) [0x7fd4e7b81b2c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7fd4e7b98468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7fd4e7b983ee]
--- /work/build/lib/libtt_metal.so(+0x57ef1f) [0x7fd4e7927f1f]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fd4e6958ee8]
--- /work/build/lib/libtt_metal.so(+0x57ed79) [0x7fd4e7927d79]
--- /work/build/lib/libtt_metal.so(+0x7ef23e) [0x7fd4e7b9823e]
--- /work/build/lib/libtt_metal.so(+0x792095) [0x7fd4e7b3b095]
--- /work/build/lib/libtt_metal.so(+0x790ba0) [0x7fd4e7b39ba0]
--- /work/build/lib/libtt_metal.so(+0x790088) [0x7fd4e7b39088]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fd4e6ccd253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fd4e6953ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fd4e69e4a84]
backtrace:
--- /work/build/lib/libtt_metal.so(+0x6df318) [0x7fd4e7a88318]
--- /work/build/lib/libtt_metal.so(+0x6be4b1) [0x7fd4e7a674b1]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /work/build/lib/libtt_metal.so(+0x6bde26) [0x7fd4e7a66e26]
--- /work/build/lib/libtt_metal.so(+0x7ef468) [0x7fd4e7b98468]
--- /work/build/lib/libtt_metal.so(+0x7ef3ee) [0x7fd4e7b983ee]
--- /work/buil
|
|
cpp-unit-tests (wormhole_b0, N150) / All C++ wormhole_b0 N150
Process completed with exit code 1.
|
|
cpp-unit-tests (blackhole, P150b) / tools blackhole P150b
Process completed with exit code 1.
|
|
cpp-unit-tests (wormhole_b0, N150) / tools wormhole_b0 N150
Process completed with exit code 1.
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 llk
Process completed with exit code 139.
|
|
cpp-unit-tests (wormhole_b0, N300) / tools wormhole_b0 N300
Process completed with exit code 1.
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 debug_tools and device and dispatch
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_matmul_1d_gather_in0.py#L601
test_multi_core_matmul_1d_wh[3-True-None-3-32-2304-3840-DataType.BFLOAT16-DataType.BFLOAT8_B-MathFidelity.HiFi2-True-False-grid4-no_bias]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation_gathered TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation_gathered.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fefd0ae3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fefce34480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fefce35b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fefce35b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fefce0eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7ff0153c1ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fefce0edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fefce35b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fefce2fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fefce2fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fefce2fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7ff012af2253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7ff0153bcac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7ff01544da84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fefce24aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fefce22ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fefce22a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/bu
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / sweep wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / sweep wormhole_b0 N150:
tests/tt_eager/python_api_testing/sweep_tests/pytests/helper_funcs/test_linear.py#L27
test_linear_no_bias[input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f84e3101168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f84e096280c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f84e09792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f84e097923e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f84e070bdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f85278c7ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f84e070bc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f84e097908e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f84e091bd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f84e091a890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f84e0919d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f8525079253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f85278c2ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f8527953a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f84e0868ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f84e0848b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f84e08484cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f84e09792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f84e097923e]
|
|
cpp-unit-tests (wormhole_b0, N150) / dispatch multicmd queue wormhole_b0 N150
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_rotary_embedding_llama.py#L493
test_rotary_embedding_llama[0.9997-DataType.BFLOAT16-11-3-128-decode_8]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f92398e3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f923714480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f923715b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f923715b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9236eeddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f927e195ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9236eedc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f923715b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f92370fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f92370fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f92370fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f927b8c6253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f927e190ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f927e221a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f923704aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f923702ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f923702a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f923715b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_attn_matmul.py#L49
test_attn_matmul[20-DataType.BFLOAT16-DataType.BFLOAT16-DataType.BFLOAT16]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transformer_attn_matmul TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tile_move_copy.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/matmul/attn_matmul/device/kernels/compute/transformer_attn_matmul.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fa22c2e7168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fa229b4880c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fa229b5f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fa229b5f23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fa2298f1def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fa266c51ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fa2298f1c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fa229b5f08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fa229b01d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fa229b00890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fa229affd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fa264351253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fa266c4cac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fa266cdda84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fa229a4eed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fa229a2eb51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fa229a2e4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fa229b5f2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fa229b5f23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L368
test_matmul_dram_sharded_mesh_device[device_params0-False-bf16_bf4b_bf16-32x7168x2304_mlp_w1_style-mesh_device0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L216
test_matmul_dram_sharded_single_device[bf16_bf4b_bf16-32x7168x4064_lm_head_style]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/pyth
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L216
test_matmul_dram_sharded_single_device[bf16_bf4b_bf16-32x256x7168_small_input]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L216
test_matmul_dram_sharded_single_device[bf16_bf4b_bf16-32x7168x256_small_output]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_create_qkv_heads.py#L159
test_nlp_create_qkv_heads_test[7-224-8-8-64-7-8-True-BFLOAT8_B-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh_sharded TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/transformer/split_query_key_value_and_split_heads/device/kernels/compute/transpose_wh_sharded.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9e50e01168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9e4e66280c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9e4e6792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9e4e67923e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9e4e40bdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9e956b1ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9e4e40bc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9e4e67908e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9e4e61bd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9e4e61a890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9e4e619d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9e92de2253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9e956acac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9e9573da84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9e4e568ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9e4e548b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9e4e5484cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9e4e6792b8]
---
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L216
test_matmul_dram_sharded_single_device[bf16_bf4b_bf16-32x2304x7168_mlp_w2_style]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/pytho
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_matmul.py#L216
test_matmul_dram_sharded_single_device[bf16_bf4b_bf16-32x7168x2304_mlp_w1_style]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/pytho
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_embedding.py#L56
test_embedding[False-Layout.TILE-mem_config0-DataType.BFLOAT16-DataType.UINT32-shape_pair1-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/bui
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_embedding.py#L56
test_embedding[False-Layout.TILE-mem_config0-DataType.BFLOAT16-DataType.UINT32-shape_pair0-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/bui
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_embedding.py#L56
test_embedding[True-Layout.TILE-mem_config0-DataType.BFLOAT16-DataType.UINT32-shape_pair1-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/buil
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
models/demos/deepseek_v3/tests/unit/test_embedding.py#L56
test_embedding[True-Layout.TILE-mem_config0-DataType.BFLOAT16-DataType.UINT32-shape_pair0-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/tilize/device/kernels/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f89adce3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f89ab54480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f89ab55b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f89ab2eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f89f25f0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f89ab2edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f89ab55b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f89ab4fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f89ab4fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f89ab4fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f89efcf0253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f89f25ebac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f89f267ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f89ab44aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f89ab42ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f89ab42a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f89ab55b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/buil
|
|
cpp-unit-tests (wormhole_b0, N150) / Nightly TTNN university lab examples wormhole_b0 N150
Process completed with exit code 134.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_bcast.py#L134
test_bcast[ShardOrientation.COL_MAJOR-1-1-BcastOpMath.ADD-DataType.BFLOAT8_B-DataType.BFLOAT8_B-512-1280-40-shard_grid6-ShardStrategy.BLOCK]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_h_sharded_optimised TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_h_sharded_optimised.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f3186801168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f318406280c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f31840792b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f318407923e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f3183e0bdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f31cb0b1ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f3183e0bc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f318407908e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f318401bd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f318401a890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f3184019d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f31c87e2253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f31cb0acac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f31cb13da84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f3183f68ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f3183f48b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f3183f484cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8)
|
|
cpp-unit-tests (wormhole_b0, N150) / distributed wormhole_b0 N150
Process completed with exit code 139.
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 512, 64)-MultiCQTracedModelPipelinedIOExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 512, 64)-MultiCQTracedModelOverlappedInputExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 512, 64)-MultiCQModelOverlappedInputExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 512, 64)-TracedModelExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
-
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 512, 64)-ModelExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /op
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 32, 32)-MultiCQTracedModelPipelinedIOExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 32, 32)-MultiCQTracedModelOverlappedInputExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 32, 32)-MultiCQModelOverlappedInputExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 32, 32)-TracedModelExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
models/tt_cnn/tests/test_pipeline.py#L257
test_executor_single_runs[device_params0-relu_model-shape_(1, 1, 32, 32)-ModelExecutor]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f9d0098e168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f9cfe1ef80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f9cfdf98def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f9d45169ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f9cfdf98c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f9cfe20608e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f9cfe1a8d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f9cfe1a7890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f9cfe1a6d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f9d428e3253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9d45164ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9d451f5a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9cfe0f5ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9cfe0d5b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f9cfe0d54cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f9cfe2062b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f9cfe20623e]
--- /opt
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_bcast.py#L134
test_bcast[ShardOrientation.COL_MAJOR-1-1-BcastOpMath.ADD-DataType.BFLOAT8_B-DataType.BFLOAT8_B-512-1280-40-shard_grid6-ShardStrategy.BLOCK]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_h_sharded_optimised TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_h_sharded_optimised.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f0c376af168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f0c34f1080c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f0c34f272b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f0c34f2723e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f0c34cb9def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f0c7bf64ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f0c34cb9c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f0c34f2708e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f0c34ec9d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f0c34ec8890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f0c34ec7d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f0c79695253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f0c7bf5fac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f0c7bff0a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f0c34e16ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f0c34df6b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f0c34df64cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8)
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/loss_ops/test_loss_mae.py#L41
TestMAELoss.test_loss_mae_none[memcfg0-input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/compute_kernel_api.h:20,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/sfpu_split_includes.h:180,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fd747ee3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fd74574480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd74575b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fd74575b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fd7454eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fd78c834ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fd7454edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fd74575b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fd7456fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fd7456fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fd7456fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fd789f65253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fd78c82fac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fd78c8c0a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fd74564aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fd74562ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fd74562a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd74575b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fd74575b23e]
--- /opt/venv/lib/python3.10/site-packages/t
|
|
cpp-unit-tests (wormhole_b0, N150) / dispatch wormhole_b0 N150
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_create_qkv_heads.py#L159
test_nlp_create_qkv_heads_test[7-224-8-8-64-7-8-True-BFLOAT8_B-device_params0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh_sharded TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/transformer/split_query_key_value_and_split_heads/device/kernels/compute/transpose_wh_sharded.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fc314ee3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fc31274480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fc31275b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fc31275b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fc3124eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fc3597c1ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fc3124edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fc31275b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fc3126fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fc3126fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fc3126fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fc356ef2253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fc3597bcac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fc35984da84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fc31264aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fc31262ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fc31262a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fc31275b2b8]
---
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/loss_ops/test_loss_mae.py#L41
TestMAELoss.test_loss_mae_none[memcfg0-input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/compute_kernel_api.h:20,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_unary/sfpu_split_includes.h:180,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fe7ada96168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fe7ab2f780c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fe7ab30e2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fe7ab30e23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fe7ab0a0def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fe7f2245ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fe7ab0a0c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fe7ab30e08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fe7ab2b0d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fe7ab2af890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fe7ab2aed78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fe7ef976253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fe7f2240ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fe7f22d1a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fe7ab1fded8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fe7ab1ddb51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fe7ab1dd4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fe7ab30e2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fe7ab30e23e]
--- /opt/venv/lib/python3.10/site-packages/t
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 All C++, API and user kernel path tests
Process completed with exit code 1.
|
|
cpp-unit-tests (wormhole_b0, N300) / dispatch multicmd queue wormhole_b0 N300
Process completed with exit code 139.
|
|
cpp-unit-tests (wormhole_b0, N300) / dispatch wormhole_b0 N300
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_bcast.py#L134
test_bcast[ShardOrientation.COL_MAJOR-1-1-BcastOpMath.MUL-DataType.BFLOAT16-DataType.BFLOAT8_B-512-1280-40-shard_grid6-ShardStrategy.BLOCK]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_h_sharded_optimised TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_h_sharded_optimised.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f23235cb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f2320e2c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f2320e432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f2320e4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f2320bd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f2353fd5ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f2320bd5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f2320e4308e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f2320e432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f2320e4323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f2320bd5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f2353fd5ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f2320bd5c49]
--- /opt/venv/lib/python3.10
|
|
cpp-unit-tests (blackhole, P100a) / distributed blackhole P100a
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_padding_test.py#L131
test_run_padding_and_add_test[input_tensor_shape0-output_tensor_shape0-input_tensor_start0-0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_binary.h:7,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f0ab91bf168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f0ab6a2080c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f0ab6a372b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f0ab6a3723e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f0ab67c9def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f0b07b24ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f0ab67c9c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f0ab6a3708e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f0ab69d9d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f0ab69d8890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f0ab69d7d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f0b05224253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f0b07b1fac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f0b07bb0a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f0ab6926ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f0ab6906b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f0ab69064cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f0ab6a372b8]
--- /opt/venv/lib/python3.10/site-packa
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
tests/tt_eager/python_api_testing/unit_testing/misc/test_padding_test.py#L131
test_run_padding_and_add_test[input_tensor_shape0-output_tensor_shape0-input_tensor_start0-0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_binary.h:7,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f31640e3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f316194480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f316195b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f316195b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f31616eddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f31a8a52ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f31616edc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f316195b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f31618fdd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f31618fc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f31618fbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f31a6183253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f31a8a4dac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f31a8adea84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f316184aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f316182ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f316182a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f316195b2b8]
--- /opt/venv/lib/python3
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
tests/tt_eager/python_api_testing/unit_testing/misc/test_padding_test.py#L131
test_run_padding_and_add_test[input_tensor_shape0-output_tensor_shape0-input_tensor_start0-0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for eltwise_binary_no_bcast TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/eltwise_binary.h:7,
from /work/ttnn/cpp/ttnn/operations/eltwise/binary_ng/device/kernels/compute/eltwise_binary_no_bcast.cpp:8,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f7974bc3168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f797242480c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f797243b2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f797243b23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f79721cddef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f79b94deee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f79721cdc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f797243b08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f79723ddd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f79723dc890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f79723dbd78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f79b6c0f253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f79b94d9ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f79b956aa84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f797232aed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f797230ab51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f797230a4cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f797243b2b8]
--- /opt/venv/lib/python3
|
|
cpp-unit-tests (blackhole, P100a) / tools blackhole P100a
Process completed with exit code 1.
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L1149
test_timestamped_events
assert 34304 == 0
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L1098
test_profiler_host_device_sync
assert 256 == 0
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/ttnn/tracy/test_profiler_sync.py#L23
test_with_ops
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fd357d50168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fd3555b180c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd3555c82b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fd3555c823e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fd35535adef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fd39c606ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fd35535ac49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fd3555c808e]
--- auto tt::tt_metal::detail::async<std::function<void ()> const&>(std::function<void ()> const&)
--- tt::tt_metal::launch_build_step(std::function<void ()> const&, std::vector<std::shared_future<void>, std::allocator<std::shared_future<void> > >&)
--- tt::tt_metal::JitBuildState::compile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*) const
--- tt::tt_metal::JitBuildState::build(tt::tt_metal::JitBuildSettings const*) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fd3555c82b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fd3555c823e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fd35535adef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fd39c606ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fd35535ac49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fd3555c808e]
--- /opt/venv/lib/py
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L967
test_dispatch_cores
assert 34304 == 0
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L800
test_device_trace_run
assert 256 == 0
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/ttnn/tracy/test_trace_runs.py#L30
test_with_ops[device_params={'trace_region_size': 1996800, 'dispatch_core_type': DispatchCoreType.WORKER}]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f3404219168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f3401a7a80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f3401a912b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f3401a9123e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f3401823def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f3448ac0ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f3401823c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f3401a9108e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f3401a33d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f3401a32890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f3401a31d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f344622e253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f3448abbac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f3448b4ca84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f3401980ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f3401960b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f34019604cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f3401a912b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f34
|
|
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/ttnn/tracy/test_trace_runs.py#L103
test_with_ops_multiple_trace_ids[device_params={'trace_region_size': 1996800, 'dispatch_core_type': DispatchCoreType.WORKER}]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for tilize TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_exp.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/cmath_common.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tilize.h:7,
from /work/ttnn/cpp/ttnn/kernel/compute/tilize.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/wormhole_b0/metal/common/chlkc_list.h:21,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_wormhole_b0/common/inc/sfpu/ckernel_sfpu_recip.h:10:10: fatal error: vconst_verifier.h: No such file or directory
10 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f537f8cb168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f537d12c80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f537d1432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f537d14323e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f537ced5def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f53c417bee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f537ced5c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f537d14308e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f537d0e5d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f537d0e4890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f537d0e3d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f53c18e9253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f53c4176ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f53c4207a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f537d032ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f537d012b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f537d0124cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f537d1432b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L270
test_custom_cycle_count
assert 34304 == 0
|
|
run-profiler-regression (wormhole_b0, N150) / profiler-regression:
tests/tt_metal/tools/profiler/test_device_profiler.py#L248
test_custom_cycle_count_slow_dispatch
assert 34304 == 0
|
|
cpp-unit-tests (blackhole, P100a) / Nightly TTNN university lab examples blackhole P100a
Process completed with exit code 134.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch blackhole P100a
Process completed with exit code 139.
|
|
fast-dispatch-unit-tests (blackhole, P100) / sweep blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / sweep blackhole P100:
tests/tt_eager/python_api_testing/sweep_tests/pytests/helper_funcs/test_linear.py#L27
test_linear_no_bias[input_shapes0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transpose_wh TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/transpose_wh.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/transpose/device/kernels/compute/transpose_wh.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f995addf168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f995864080c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f99586572b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f995865723e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f99583e9def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f99a95d3ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f99583e9c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f995865708e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f99585f9d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f99585f8890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f99585f7d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f99a6cce253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f99a95ceac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f99a965fa84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f9958546ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f9958526b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f99585264cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f99586572b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f995865723e]
--- /opt/venv/
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_matmul_dram_sharded.py#L254
test_matmul_in1_dram_sharded_with_program_cache[False-True-True-32-8192-1280-None-grid_size0-DataType.BFLOAT16-DataType.BFLOAT8_B-DataType.BFLOAT16-no_bias-no_packer_l1_acc-HiFi2]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bmm_large_block_zm_fused_bias_activation TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/matmul.h:7,
from /work/ttnn/cpp/ttnn/operations/matmul/device/kernels/compute/bmm_large_block_zm_fused_bias_activation.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x76375f7d2168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x76375d03380c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x76375d04a2b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x76375d04a23e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x76375cddcdef]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7637cd6ddee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x76375cddcc49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x76375d04a08e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x76375cfecd85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x76375cfeb890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x76375cfead78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7637cabdd253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7637cd6d8ac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7637cd769a84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x76375cf39ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x76375cf19b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x76375cf194cb]
--- /opt/venv/lib/python3.10/site-packag
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_attn_matmul.py#L49
test_attn_matmul[20-DataType.BFLOAT16-DataType.BFLOAT16-DataType.BFLOAT16]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for transformer_attn_matmul TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/tile_move_copy.h:7,
from /work/ttnn/cpp/ttnn/operations/experimental/matmul/attn_matmul/device/kernels/compute/transformer_attn_matmul.cpp:6,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7fdf4706d168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7fdf448ce80c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fdf448e52b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fdf448e523e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7fdf44677def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7fdfb3941ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7fdf44677c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7fdf448e508e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7fdf44887d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7fdf44886890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7fdf44885d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7fdfb0ff8253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7fdfb393cac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7fdfb39cda84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7fdf447d4ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7fdf447b4b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7fdf447b44cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7fdf448e52b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7fdf448e523e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100
Process completed with exit code 1.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
tests/tt_eager/python_api_testing/unit_testing/misc/test_rotate_half.py#L22
test_rotate_half[shape0]
RuntimeError: TT_THROW @ /project/tt_metal/impl/program/program.cpp:155: tt::exception
info:
Failed to generate binaries for bcast_hw TT_THROW @ /project/tt_metal/jit_build/build.cpp:58: tt::exception
info:
trisc1 build failed. Log: In file included from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_exp.h:11,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_activations.h:13,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/ckernel_sfpu.h:14,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/cmath_common.h:12,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_math_common_api.h:10,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common_globals.h:16,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/common.h:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/inc/api/compute/bcast.h:7,
from /work/ttnn/cpp/ttnn/operations/data_movement/bcast/device/kernels/compute/bcast_hw.cpp:7,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/ckernels/blackhole/metal/common/chlkc_list.h:22,
from /opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/hw/firmware/src/tt-1xx/trisck.cc:13:
/opt/venv/lib/python3.10/site-packages/ttnn/tt_metal/third_party/tt_llk/tt_llk_blackhole/common/inc/sfpu/ckernel_sfpu_recip.h:13:10: fatal error: vconst_verifier.h: No such file or directory
13 | #include "vconst_verifier.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/_ttnncpp.so(+0x1f16168) [0x7f4e768df168]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7c480c) [0x7f4e7414080c]
--- tt::tt_metal::JitBuildState::compile_one(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tt::tt_metal::JitBuildSettings const*, unsigned long) const
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e741572b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4e7415723e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56ddef) [0x7f4e73ee9def]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x99ee8) [0x7f4ec5223ee8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x56dc49) [0x7f4e73ee9c49]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db08e) [0x7f4e7415708e]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77dd85) [0x7f4e740f9d85]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77c890) [0x7f4e740f8890]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x77bd78) [0x7f4e740f7d78]
--- /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253) [0x7f4ec2923253]
--- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f4ec521eac3]
--- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f4ec52afa84]
backtrace:
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6caed8) [0x7f4e74046ed8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aab51) [0x7f4e74026b51]
--- tt::tt_metal::JitBuildCache::build_once(unsigned long, std::function<void ()> const&)
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x6aa4cb) [0x7f4e740264cb]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db2b8) [0x7f4e741572b8]
--- /opt/venv/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x7db23e) [0x7f4e7415723e]
--- /opt/venv/lib/python3.10/site-packages/
|
|
tt-train-cpp-unit-tests (wormhole_b0, N150) / tt-train wormhole_b0 N150
Process completed with exit code 8.
|
|
sd-unit-tests (blackhole, P100) / blackhole P100 debug_tools and device and dispatch
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 6 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 llk
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
sd-unit-tests (blackhole, P100) / blackhole P100 llk
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 7 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 7 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 5 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 4 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (blackhole, P150b) / Nightly TTNN university lab examples blackhole P150b
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 3 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 debug_tools and device and dispatch
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 2 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 llk
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 All C++, API and user kernel path tests
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
sd-unit-tests (blackhole, P100) / blackhole P100 All C++, API and user kernel path tests
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
sd-unit-tests (blackhole, P150) / blackhole P150 debug_tools and device and dispatch
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
cpp-unit-tests (wormhole_b0, N300) / distributed wormhole_b0 N300
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 6 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 3 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn builder unit tests wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
cpp-unit-tests (blackhole, P150b) / dispatch blackhole P150b
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 4 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
sd-unit-tests (wormhole_b0, N300) / wormhole_b0 N300 All C++, API and user kernel path tests
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
cpp-unit-tests (blackhole, P150b) / dispatch multicmd queue blackhole P150b
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 7 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 7 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (blackhole, P150b) / distributed blackhole P150b
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
cpp-unit-tests (wormhole_b0, N300) / Nightly TTNN university lab examples wormhole_b0 N300
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 llk
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 debug_tools and device and dispatch
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 4 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (wormhole_b0, N150) / dispatch multicmd queue wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 6 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 1 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 3 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
models-unit-tests (wormhole_b0, N150) / model wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (wormhole_b0, N150) / Nightly TTNN university lab examples wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 2 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (wormhole_b0, N150) / distributed wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
tt-cnn-unit-tests (wormhole_b0, N150) / tt-cnn pipeline unit tests wormhole_b0 N150:
/work/models/tt_cnn/tests/test_pipeline.py#L27
cannot collect test class 'TestShapeConfig' because it has a __init__ constructor (from: models/tt_cnn/tests/test_pipeline.py)
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 2 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 1 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (wormhole_b0, N150) / dispatch wormhole_b0 N150
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 3 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 1 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
sd-unit-tests (wormhole_b0, N150) / wormhole_b0 N150 All C++, API and user kernel path tests
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
cpp-unit-tests (wormhole_b0, N300) / dispatch multicmd queue wormhole_b0 N300
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
cpp-unit-tests (wormhole_b0, N300) / dispatch wormhole_b0 N300
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P150) / eager unit tests 2 blackhole P150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
cpp-unit-tests (blackhole, P100a) / distributed blackhole P100a
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 5 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N150) / eager unit tests 5 wormhole_b0 N150:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (wormhole_b0, N300) / eager unit tests 5 wormhole_b0 N300:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
cpp-unit-tests (blackhole, P100a) / Nightly TTNN university lab examples blackhole P100a
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
cpp-unit-tests (blackhole, P100a) / dispatch blackhole P100a
No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 4 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 1 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
|
|
mlperf-mount-service-not-found-startup
MLPerf mount service not found.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_fields.py#L132
Field "model_name" in OpTest has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyObject has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
<frozen importlib._bootstrap>#L241
builtin type SwigPyPacked has no __module__ attribute
|
|
fast-dispatch-unit-tests (blackhole, P100) / eager unit tests 6 blackhole P100:
/opt/venv/lib/python3.10/site-packages/pydantic/_internal/_config.py#L291
Support for class-based `config` is deprecated, use ConfigDict instead.
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
test_reports_01e119c5-7ea9-45dd-bbf4-dd77aaabedc7
|
3.37 KB |
sha256:04e82d85d207ffb64acbb389bd093b7a6a50f209c07300cb178f23d0d95b8067
|
|
|
test_reports_0f780220-2f1f-4b1b-a1ab-747fd528b4a8
|
3.41 KB |
sha256:c67da9cafd4c4e2c686c86a4b64334a6d0c30fc75bb24fcd49203e00cfdec0da
|
|
|
test_reports_25d8c455-6518-40d2-9b69-43290200251b
|
3.36 KB |
sha256:a250af59ee3cade26f5f1757382e926c7d733f5eeff3e545836faa50f9c2eede
|
|
|
test_reports_2743e4e7-3b33-4d38-b47b-dd2e5a7528b6
|
1.79 KB |
sha256:f88e912300fb889d622a58d2e6b82e35b42fdc5e25676dcdf000cf2d3aad0ca1
|
|
|
test_reports_2c96394c-4e91-4e12-8971-c7aea89b48d6
|
534 Bytes |
sha256:7b4e094fc7f8aebb02ff5d43236d12ad4c98833178f4e78b4065057c7a69f302
|
|
|
test_reports_2dca788d-59b7-4541-abeb-0d0459e7d41c
|
3.92 KB |
sha256:1d58211f5be984ed00fd494000a71f308f5b8bbe264c78ec19212275e4213b55
|
|
|
test_reports_303d3fdd-3498-4d0a-8a61-1303906bbafc
|
6.96 KB |
sha256:9f7c8961fcf3385cc0d7e74605b3aff3dc4f850406eed106501da9f840ae0587
|
|
|
test_reports_31ddcf8a-c1a8-4185-a065-969d465eefe1
|
3.61 KB |
sha256:774b62cb72b379e84278283c3a680d0f9cc31816f247ef3e997dc9a080109946
|
|
|
test_reports_39a3add0-da10-4403-a420-a4ba57603f69
|
4.3 KB |
sha256:295316db5da5b99348eb5489bf1937706b753f2186f2ec38f5ed51bb7ba508e9
|
|
|
test_reports_3abb4ef8-13cb-46d3-a8bf-3bd0b8a2f8ec
|
533 Bytes |
sha256:519d3ec591114aa6ac816733eb2a501b8eba4a80459cb169087e5c2f6dd0e59a
|
|
|
test_reports_4fd7ab7b-6909-4068-b6e8-7c9951fba8d1
|
1.79 KB |
sha256:175e6158a9901adb1b9bc619c9d227cdb68b4418d18a3fb0446009c505a8d82c
|
|
|
test_reports_534a3936-afd3-4492-b56c-5d286cb126c3
|
3.3 KB |
sha256:d033fccc8cf903aab501f2fba9cac11d6f0fd76e687286b5cb1d76c41dba0822
|
|
|
test_reports_53c9ece3-98bd-4441-b5fa-942c9c823aaf
|
2.77 KB |
sha256:07a9b96ab8774c8258cfbcd34df4a406e240e4c2b0d95d860ec536576b96d029
|
|
|
test_reports_545b0058-99f5-49a3-8f05-0738a05a6839
|
2.77 KB |
sha256:1b3d6e6b036164e6a4144ae5d03982fee412f15b9f2bf1b7fecd9eb44d4835d2
|
|
|
test_reports_56097073-2323-4ae1-8e9d-12c54faf4539
|
3.08 KB |
sha256:e4878b5d456cf086feb909cc78bd938f6cce251dfd6608bcfb516f53ac3b52e2
|
|
|
test_reports_57d601af-8c49-43c4-a479-8406e69184ea
|
18.2 KB |
sha256:e6fb62d5216f8608ff713d82999ae1697290348a27acaf48a9b0dc0604448f29
|
|
|
test_reports_5926f722-a61f-45f6-89be-aff035d0f726
|
2.38 KB |
sha256:97c9e5e2af24fd0eccaea1463e895a20524402179c7762d5e4f2cadb6371dd91
|
|
|
test_reports_6249a843-e8fa-46f3-8d7d-5c933c362188
|
2.96 KB |
sha256:6b3db9d6fd558bf01cab952d340ba6c249c5568ec42eacc89f21babe55a7b86a
|
|
|
test_reports_6318084b-9132-4d1b-b976-27d493305330
|
7.5 KB |
sha256:9e18f5fd091917446316eae69f616f88c5a1c7c09e0f78eb1676e45b8cb00025
|
|
|
test_reports_7e29d86c-0b7f-43db-9860-444dc92836a5
|
3.93 KB |
sha256:99cf647eb2681f52f2379fa4cb4ccc2b3da8a6d3882c7110c57d553de2c9a86b
|
|
|
test_reports_88807508-c48a-4554-b348-4bfa445334c4
|
6.75 KB |
sha256:b8da3c504a87fda5faa14f8b969d2c03573b8b895f384e47e0c4a497779ae719
|
|
|
test_reports_8903b352-c464-42b7-bcc7-4037a68314e0
|
3.61 KB |
sha256:1cf99c3dc8a49e3c1b044a360985cad4598c488fa72ed181cb16ed6e722fe39d
|
|
|
test_reports_8916695d-0f9d-43b0-990c-6ee3ab683a6a
|
3.53 KB |
sha256:ee6038b03390aaadc35d7a9c725cdd07ce29bba61bf55a2035130eda140b11ee
|
|
|
test_reports_9075c781-7d6a-4f92-a8e8-8f4147aa0c2e
|
4.16 KB |
sha256:0721468acaa2623b6d2c6c5750e6e706378bb40a6c91012a191de41fd0bf49e5
|
|
|
test_reports_a0f886f4-a97a-4859-ab73-ec3887a70488
|
3.63 KB |
sha256:c734f3ae808f0742425485e416d5b4d3999d5e7c8c97e4b410b4133a5f31790e
|
|
|
test_reports_a281e699-a8cc-46e4-b30d-7cd142c88af4
|
7.18 KB |
sha256:535d0de7fb76aa1021bfc49a7c3b90c618dfc050e9653746f439a65ee82bb715
|
|
|
test_reports_aafe0ed7-f2b8-433b-a955-43828ffd6cb5
|
12.4 KB |
sha256:36cf4e80ee9e0a22d2a33cda7538fa3bab63134cf30cd97d438be6ca6d39b4f8
|
|
|
test_reports_adafef7e-30fc-4d46-aa3e-3d639fbd7a65
|
3.17 KB |
sha256:cffa12870740a32ceb75caeee6f10e62a1781ec39793dfee2c329dba4c069533
|
|
|
test_reports_adc761b7-da1a-47de-b178-0eb77893058c
|
165 KB |
sha256:16e71df7010017bd400d3f137fca83fa653031207e4f721790c1e8caa4e32e4f
|
|
|
test_reports_ae6ca934-7a86-4404-9747-512b9306042b
|
535 Bytes |
sha256:f9b5cb88f232b29993d906d477a9dbe940f8b3cde9142dba81adf775252c8db0
|
|
|
test_reports_b29669e6-6c64-42d4-912d-7ff1b25149e3
|
3.8 KB |
sha256:fc0af3c6d5cec45cda09bfd4a78dec82ec5adee50bef7247077691736d6d53c7
|
|
|
test_reports_b3992f1f-cc52-420d-ab40-2c3903e58bac
|
1.78 KB |
sha256:91c6f2642bc047c5aa80ccd561759245b318d946e6b05ce0474639036b152491
|
|
|
test_reports_bd5f26fa-d018-4392-bad2-2633db92be2c
|
6.58 KB |
sha256:b780f8a3d887ade351735359940982af668687b7006a61a74a75ccca3d9c07cd
|
|
|
test_reports_be38942d-2dff-4bf8-8228-c8921daa4c64
|
2.77 KB |
sha256:f26b6031e48fcfd661f651a1dc8f1692103bcc21aae67ba876f227f7f645e98f
|
|
|
test_reports_c1bc0787-69b6-4d60-9e85-7f231e97a9ce
|
7.57 KB |
sha256:a856363049be35395d12a90a9b3eebc08aaf01f48de7f0127dbf731e030001ea
|
|
|
test_reports_c22cc4ef-10ec-4ca9-842e-1363ea686394
|
7.19 KB |
sha256:0c298a6229fac585b92bb414013f7273f2b7efdccf7aa7f3cf5943cc720dd580
|
|
|
test_reports_d10f0d50-ba9a-4be5-9c66-d3a9e8cb4d00
|
3.09 KB |
sha256:7990bcab78112269951076690da4b5f61c5b8d29181b4b997450a5d785fc6da0
|
|
|
test_reports_d51453ea-88c0-44f1-84a2-525944e0cd82
|
1.79 KB |
sha256:4a13a2ae270aac1bdd5172c822fb63c64a64fa87e8a29aa265d56c06ddc07c45
|
|
|
test_reports_db4bc7c4-6327-4345-9690-213a3e323cb7
|
2.77 KB |
sha256:8ebafb4e6e7418632f1a514c57fd182a8e6baec82e018600aca7c3dde2374d87
|
|
|
test_reports_e164da35-699a-46a3-af1e-6853260684eb
|
2.39 KB |
sha256:3aa1dc52559c01df7abfda0a0d3bbb691c36211b53a9a685278e1c28a1349c01
|
|
|
test_reports_e7de778e-1a4c-4d3c-b2aa-434898dd7f08
|
3.79 KB |
sha256:3069fea3725da49f5c0b57f4d325480fdc69221c7b89c0f66527283a20c4a45d
|
|
|
test_reports_f730b751-b27f-4076-9dfc-ba7005e49256
|
535 Bytes |
sha256:1ce66de0176d515506f50a3742a96e254f6a730eddcc637f6ed678c862e38acd
|
|
|
test_reports_fab6e9e6-722f-4fbf-a837-e5a6a9282073
|
7.58 KB |
sha256:5e2afabee527b00d5714eaaf949a206563892747a5358804efc31c84b6a4aa6f
|
|
|
test_reports_fac0af18-3d65-4489-9981-1335a4d35b81
|
3.75 KB |
sha256:bac9f6169df82d2243188b76c0ecb6fc7ac8b14676b4783ba23199b616490421
|
|