LocalAI version:
v3.9.0
Environment, CPU architecture, OS, and Version:
hipBLAS container image
Describe the bug
Many models (especially MoE one) fails to load on gfx906 GPUs due to lack of TensileLibrary.dat. Relevant line from log:
DEBUG GRPC stderr id="Nemotron-3-Nano-30B-A3B-127.0.0.1:43781" line="rocBLAS error: Cannot read /opt/rocm-6.4.3/lib/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx906" caller={caller.file="/build/pkg/model/process.go" caller.L=146 }