File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -72,6 +72,13 @@ benefit of letting you run llamafiles greater than 4GB on Windows.
7272On Linux, NVIDIA users will need to install the CUDA SDK (ideally using
7373the shell script installer) and ROCm users need to install the HIP SDK.
7474They're detected by looking to see if ` nvcc ` or ` hipcc ` are on the PATH.
75+ For AMD systems, make sure the executable directory containing ` hipcc ` is
76+ on your ` PATH ` and that it can be executed by your user; a `hipcc:
77+ Permission denied` message means ROCm was found but can't be run, so GPU
78+ offload will not be available until the SDK permissions or installation
79+ are fixed. Running with ` --gpu amd ` or ` --gpu nvidia ` is a useful way to
80+ turn an otherwise quiet CPU fallback into an explicit startup error while
81+ you diagnose the toolchain.
7582
7683If you have both an AMD GPU * and* an NVIDIA GPU in your machine, then
7784you may need to qualify which one you want used, by passing either
You can’t perform that action at this time.
0 commit comments