Skip to content

Commit 18db09f

Browse files
authored
Clarify Linux GPU offload diagnostics (#967)
Co-authored-by: adminlip <adminlip@users.noreply.github.com>
1 parent 3dd2005 commit 18db09f

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

docs/support.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,13 @@ benefit of letting you run llamafiles greater than 4GB on Windows.
7272
On Linux, NVIDIA users will need to install the CUDA SDK (ideally using
7373
the shell script installer) and ROCm users need to install the HIP SDK.
7474
They're detected by looking to see if `nvcc` or `hipcc` are on the PATH.
75+
For AMD systems, make sure the executable directory containing `hipcc` is
76+
on your `PATH` and that it can be executed by your user; a `hipcc:
77+
Permission denied` message means ROCm was found but can't be run, so GPU
78+
offload will not be available until the SDK permissions or installation
79+
are fixed. Running with `--gpu amd` or `--gpu nvidia` is a useful way to
80+
turn an otherwise quiet CPU fallback into an explicit startup error while
81+
you diagnose the toolchain.
7582

7683
If you have both an AMD GPU *and* an NVIDIA GPU in your machine, then
7784
you may need to qualify which one you want used, by passing either

0 commit comments

Comments
 (0)