Bug: 0.10.0 release breaks GPU on older OSs
bughigh severity
### Contact Details
_No response_
### What happened?
Previously, it worked with GPU acceleration:
```
user@pop-os:~$ ~/.local/opt/llamafile/llamafile-0.9.3 -m ~/llms/tinyllama-1.1b-chat-v1.0.Q2_K.gguf --server --host '::' --port 8081 --log-disable -ngl 999
{"function":"server_params_parse","level":"INFO","line":2737,"msg":"logging to file is disabled.","tid":"12988640","timestamp":1775857540}
import_cuda_impl: initializing gpu module...
get_rocm_bin_path: note: hipcc not found on $PATH
get_rocm_bin_path: note: $HIP_PATH/bin/hipcc does not exist
get_rocm_bin_path: note: /opt/rocm/bin/hipcc does not exist
link_cuda_dso: note: dynamically linking /home/user/.llamafile/v/0.9.3/ggml-rocm.so
link_cuda_dso: warning: libamdhip64.so.6: cannot open shared object file: No such file or directory: failed to load library
link_cuda_dso: note: dynamically linking /home/user/.llamafile/v/0.9.3/ggml-rocm.so
link_cuda_dso: warning: libamdhip64.so.6: cannot open shared object file: No such file or directory: failed to load library
get_nvcc_path: note: nvcc not found on $PATH
get_nvcc_path: note: $CUDA_PATH/bin/nvcc does not exist
get_nvcc_path: note: /opt/cuda/bin/nvcc does not exist
get_nvcc_path: note: /usr/local/cuda/bin/nvcc does not exist
link_cuda_dso: note: dynamically linking /home/user/.llamafile/v/0.9.3/ggml-cuda.so
ggml_cuda_link: CUDA kernel version 12.7
ggml_cuda_link: CUDA runtime version is 12.8 (!!!)
ggml_cuda_link: welcome to CUDA SDK with tinyBLAS
---snip---
```
The new version doesn't:
```
user@pop-os:~$ ~/.local/opt/llamafile/llamafile-0.10.0 -m ~/llms/tinyllama-1.1b-chat-v1.0.Q2_K.gguf --server --host '::' --port 8081 --log-disable -ngl 999
cuda: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/user/.llamafile/v/0.10.0/ggml-cuda: failed to load library
cuda: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/user/.llamafile/v/0.10.0/ggml-cuda: failed to load library
warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions
```
(No further output; at this point I can interact with it, but nvtop shows no GPU use and tokens/second shows it much slower in a way that I would expect on CPU)
And indeed my system doesn't have GLIBCXX_3.4.32:
```
user@pop-os:~$ grep PRETTY /etc/os-release
PRETTY_NAME="Pop!_OS 22.04 LTS"
user@pop-os:~$ ls /lib/x86_64-linux-gnu/libstdc++*
/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libstdc++.so.6.0.30
```
(This version is a bit old, but it's still supported, and the next release was only quite recently released: https://endoflife.date/pop-os )
### Version
llamafile v0.10.0
### What operating system are you seeing the problem on?
Linux
### Relevant log output
```shell
```
0 条评论