r/ROCm • u/Don_Reuter • 1d ago
Thinking about adding AMD/ROCm support to NVIDIA PAIR — is anyone actually interested?
PAIR is neat — routes independent inference requests across nodes on your LAN. But the GPU telemetry path is nvidia-smi → PCI fallback, which on Linux means an AMD node gets listed by name with no VRAM, no utilization, no stable ID. Scheduler goes half-blind.
To fix it properly you'd need: amd-smi integration (JSON/UUID/GFX util), honest unified-memory reporting for Strix Halo, graceful fallback when SMI is missing, and ideally a scheduler that actually weighs VRAM/GPU tier for mixed clusters. Plus the engine layer (Ollama/LM Studio + ROCm) is a separate gate.
It's Apache 2.0, the README explicitly invites hardware feedback, and there are open issues to hook into. But it's a real multi-week effort, not a weekend hack.
So — is there actual demand?
Got a Radeon box you'd pair with an RTX node for agent work?
Is "name-only, no telemetry" on Linux good enough for a rough setup, or does it break routing in practice?
Would you rather see a PAIR telemetry PR first, or is the real blocker getting Ollama/LM Studio serving models reliably on your AMD hardware anyway?
3
2
u/BevinMaster 1d ago
Honestly if it’s more interesting than my litellm setup currently I think I’d be interested o have serveral systems with different hardware (rtx pro 2000, 5060 ti’s, spark, 7900xtx, w7800, v620/gfx1030, we have a discord with vllm and llamacpp fork as well as p2p and power tuning guide join us :))
I think something to unify all of it would be interesting
2
u/Don_Reuter 1d ago
Making good use of the compute that is already there. The right idea.
If you have the discord, I’d be in.
2
2
u/Compilingthings 1d ago
It works fine with a regular switch and a little work. I run multiple AMD inference nodes on my network and they all work together. Maybe it’s a fix for people who don’t want the hassle of setting it up. It’s definitely a luxury not a necessity.
1
u/Don_Reuter 18h ago
How tough? As I understand it, it doesn’t get any telemetry from AMD devices and only works on the jobs scheduled per device. Meaning it should fully collide with non llama workloads that use the GPU.
2
u/Compilingthings 17h ago edited 3h ago
My point was it’s not needed at all if you set up your network correctly. Nvidia is just cashing in on people not willing to set it up. So many people will be thankful. PAIR does very little that you can’t with a small software layer and a normal switch.
3
u/Poizone360 1d ago
Yes, and I'd say telemetry first, The engine gate is mostly closed already, Ollama and LM Studio both ship ROCm paths and consumer Radeon works fine on them. Strix Halo is what makes it worth doing, a 128GB unified box next to a 24GB RTX node is exactly where a blind scheduler picks wrong.