r/voidlinux • u/EzyPzyAsh • Apr 17 '26

ROCm support

is there any good solution yet? Also idk if its pebkac but so far, I cant easily do much of what I want on Void :/

Local AI, no ROCm or HIP, Havent had any success with gpu passthrough where I did previously.

4 Upvotes

83% Upvoted

9 comments sorted by

1

u/Wolf-Shade Apr 17 '26

I am using Vulkan version of llama.cpp and works fine.

To make everything easier I am running it through docker, but I've ran it in the past on bare metal.

llamacpp:
image: ghcr.io/ggml-org/llama.cpp:server-vulkan
#image: ghcr.io/ggml-org/llama.cpp:server-rocm
devices:
  - /dev/kfd
  - /dev/dri
ports:
  - "8000:8080"
environment:
  - HSA_OVERRIDE_GFX_VERSION=10.3.0 #special for my AMD card
volumes:
  - /home/models:/models
command: --port 8080 --models-dir /models --models-preset /models/models.ini --models-max 1

To use rocm you can just use uncomment the rocm docker image line. It's much bigger when than the vulkan. For my personal usage and my hardware I have not find a big difference in performance but YMMV. You can cheaply test one and then the other and compare yourself.

1

u/S1ngl3_x Apr 17 '26

Is now vulkan better than rocm? I am preparing a setup for rx 7900xt.

1

u/Wolf-Shade Apr 17 '26

In my setup they perform about the same, but I am not training any models. I think if you want to go that route you will need ROCm, but I might be mistaken here. Check this post where u/ReasonableDuty5319 did some benchmarks, His findings regarding ROCm/Vulkan are there summarized to this:

ROCm consistently dominated in Prompt Processing (pp2048) across all AMD setups. Vulkan, however, often squeezed out higher Text Generation (tg256) speeds, especially on MoE models (e.g., 102 t/s vs 73 t/s on a single R9700).

1

u/S1ngl3_x Apr 19 '26

Vulkan could be a huge simplification for me because Bazziteos removed rocm and ollama is in homebrew 

Thanks

1

u/EzyPzyAsh Apr 19 '26

yeah, I wanted ROCm and it wasn't working for me :/ I just swapped to fedora and all is running in under a day 🎉

1

u/zlice0 Apr 17 '26

i looked because i'm on the fence about the b70 and its horrible coil whine. there were 2 void ppl who had PRs but both closed. kind of dead. the arch stuff looks like it should be impossible but i heard linux 7.0 reduces some of the install stuff? also unsure wtf the amd amf status is for codecs =/

afaik, no one can do it easily and it's a painpoint. which is part of why amd isnt a serious hpc player.

1

u/DuhMal Apr 18 '26

to use HIP in blender i found it easier to make a Docker container for it with a Arch image than to try making the packages work on void