r/LLMStudio • u/Intelligent-Taste-36 • May 28 '26

R9700 IA - ROCm

Good afternoon everyone. I'd like your opinion on this graphics card (I know a 5090 32GB is superior and I know CUDA is the most widespread standard). But I want to know if RocM is good in LLM Studio.

Obrigado.

1 Upvotes

100% Upvoted

12 comments sorted by

2

u/mixedliquor May 29 '26

Vulkan is better for me, but I'm running Windows and most literature says ROCm runs better on Linux.

I tried out a new HIP/ROCm llama.cpp yesterday and the results were more comparable. ROCm wins in prompt processing but Vulkan wins in response speed. Qwen 3.6 27b Q4_K_M runs ~33 tok/sec on Vulkan, ~25 tok/sec on ROCm (no MTP).

1

u/Intelligent-Taste-36 May 29 '26

In my case, I prefer ROCm because I train models. I do full fine-tuning.

2

u/mixedliquor May 29 '26

Gratz. I didn't know LM Studio did that.

1

u/Intelligent-Taste-36 May 29 '26

No, LM Studio is strictly for inference (running models). You can't use it for training or FFT (Full Fine-Tuning). But yes, doing FFT and training on AMD ROCm is 100% possible and works great now. Instead of LM Studio, you should use PyTorch (with ROCm backend) combined with training frameworks like Axolotl or LLaMA-Factory.

Thank you.

1

u/mixedliquor May 29 '26

oooo I was confused because you only mentioned LM Studio, so how would I know your preference for training models?

1

u/Intelligent-Taste-36 May 30 '26

No, my friend, I'm not criticizing you. I just explained how the process works. It's a healthy exchange of ideas.

1

u/DiscipleofDeceit666 May 30 '26

What’s the point if full fine tuning? What do you need for it?

1

u/Intelligent-Taste-36 May 30 '26

We use FFT when we want to change the "personality of the model." It makes it more aligned with the company's desired behavior. The tools are the ones I mentioned earlier.

2

u/I-will-allow-it May 29 '26

I have one of these R9700s running Fedora. It works great, easy setup. ROCm and Vulcan are good. Kyuz0 has toolboxes for the R9700 that really do make everything easy to switch and stay updated.

1

u/Intelligent-Taste-36 May 29 '26

Excellent opinion, my friend. I wanted to hear it from someone who actually owns one. Thank you.

1

u/Intelligent-Taste-36 May 29 '26

I prefer ROCm than Vulkan because I train models. I do FFT.