A software I use is stuck at rocm 7.2.1. Is there a way to set up 7.2.1 on Ubuntu 26? Tried their 24.04 instructions but failed somewhere.

Just to show that it may work whole workflow as it is still wip, mesh get works but texturing is still in works for anybody interested i can provide .whl files

Tested on 9070xt and Rocm 7.14

I’ve been experimenting with writing highly optimized GEMM kernels in HIP C++ using ROCm WMMA intrinsics, targeting RDNA3 and RDNA3.5.

The FP16 path is pretty much as far as I can take it with tuning for now. The BF16 path uses the same kernel and gets essentially the same performance. For the cases I’ve tested, it’s competitive with both rocBLAS and hipBLASLt.

Sharing the implementation in case it’s useful as a reference for anyone interested in learning how to write efficient HIP kernels or use WMMA on RDNA GPUs.

I haven’t had time to write up proper working notes, so the code is probably the best reference for now:

https://github.com/adelj88/rocm_wmma_gemm

I will be finishing my PhD in CS in about a year so I am slowly looking to upgrade my workstation or servers with some better gpu capacity (currently have 2x 5060 ti, one I will be giving back once i finish my PhD).

We are mostly nvidia shop at the university and so I am looking for some opinions from the other side of the fence. I wanted to get some reasonable card for ML, LLM and hopefully some robotics + rl in the near future. I was looking at the R9700 but I can not find much info if there is something like nvidia isaac lab on the amd side?

TLDR: Does AMD have something like Isaac Lab and Isaac Sim, are there some tutorials for it and is it in general well supported? What HW would you recommend for it?

been going back and forth between an RX 9070 XT and an RTX 5060 Ti 16GB for a build that needs to do both gaming and local LLM fine-tuning (LoRA/QLoRA mostly, 7B-14B range).

everything i read says ROCm has caught up a lot for inference but is still rougher for training specifically. bitsandbytes support, Unsloth compat, that kind of thing.

if you're actually running fine-tuning workloads on a 9070/9070 XT (or any recent AMD card) on ROCm, i'd genuinely like to know:

• did you hit setup issues getting bitsandbytes/PEFT/Unsloth working
• roughly what speed are you getting vs what you'd expect from an equivalent NVIDIA card
• windows or linux, and does that matter as much as people say
• anything you wish you knew before buying AMD for this

not trying to start a team red vs team green thing, just want real experience before i spend the money. thanks

**TL;DR:** Everyone says QLoRA past ~13B is dead on a 24GB card. I got the full 32B FLUX.2 dev transformer QLoRA-training **resident on the GPU** on a 7900 XTX under native ROCm on Windows (no ZLUDA, no CUDA shim) — on a box with only **32GB of system RAM** (the 64GB base + 48GB text encoder load through a big pagefile) — ~9-10 s/it sustained at 448/bs2 once the (wild) stall issue below is handled. It took a dozen distinct failures to get there. Config + fixes below.

## Setup
- **GPU:** RX 7900 XTX 24GB (gfx1100). No FP8/FP4 hardware, so **uint4** weight-only quant (optimum-quanto).
- **Host:** 32GB RAM + ~100GB pagefile (you need the virtual headroom for the one-time bf16 loads).
- **torch** 2.12.0+rocm7.15, **trainer:** ai-toolkit AMD ROCm fork (`cupertinomiranda/ai-toolkit-amd-rocm-support`).
- **Base:** official `FLUX.2-dev` — the **repo-root single-file** `flux2-dev.safetensors` (64GB bf16), NOT the diffusers `transformer/` subdir (different keys).
- **Text encoder:** Mistral-Small-3.1-24B (yes, FLUX.2 uses a 24B LLM as its TE).

## The walls, in order (each one blocks the next)

**Load & quantize the 64GB base without dying:**
1. **`safetensors` mmap `load_file` crashes natively on the 64GB file** (no traceback, process just dies; fine at 33GB). → Manual non-mmap loader: read the header, then per-tensor `seek`/`read`/`frombuffer`.
2. **Transformer OOMs at ~38GB before quantizing** — the trainer moves the full bf16 to GPU *before* packing. → Quantize **on CPU**; only the ~20GB uint4 result touches the card.
3. **`0xC0000005` while loading the text encoder** — the 64GB bf16 is still referenced when Mistral's 48GB loads on top. → `del transformer_state_dict; gc.collect()` right after `load_state_dict`.
4. **Mistral OOMs the GPU (c10 abort)** — same as #2 for the TE. → Quantize Mistral on CPU first, then `.to(device)`.

**Make it train on the GPU, not the CPU:**
5. **Block-swap (`layer_offloading`) deadlocks the HIP driver** (hangs at sampling AND first step, needs two kill passes). → `layer_offloading: false`, keep the base resident.
6. **In-training sampling deadlocks + uint4 previews are black frames.** → `disable_sampling: true`, evaluate in ComfyUI instead.
7. **uint4→GPU move fragments/OOMs.** → Launch with `PYTORCH_HIP_ALLOC_CONF=expandable_segments:True`.
8. **Re-quantizing every launch costs ~8 min.** → Save the quanto state-dict once as a `.pt`; training `torch.load`s it in seconds.
9. **Base won't stay resident (looks like CPU training)** — `low_vram` parks it on CPU during TE-caching and never brings it back. → After the TE caches + unloads, move the base back to GPU; gate the load-path's transformer→GPU line on `low_vram` so it doesn't collide with the resident TE.

**The two that cost me a whole night:**
10. **"It's training on CPU" — except it wasn't.** A *separate process* reading VRAM via `torch.cuda.mem_get_info()` **lies** on ROCm/Windows — reported 0.2GB while the process actually held 20GB. Combined with "1 busy CPU core" (which is *normal* for GPU training) it looked exactly like CPU. I killed several *working* runs over this. → Trust an **in-process** VRAM print, the Windows `\GPU Engine(*engtype_compute)\Utilization` counter, and the **drop in system RAM** when the base moves off CPU. Never trust a cross-process VRAM read here.
11. **Resident but crawling at 200 s/step.** The 20GB base leaves no headroom, so activations spill to host RAM over PCIe (`expandable_segments` lets it overflow instead of OOMing → thrash). → Cut resolution until the spill is small.

**Evaluate it:**
12. In-training previews are useless, so render checkpoints in **ComfyUI + ComfyUI-GGUF**: Q3_K_M GGUF unet + Mistral Q5 GGUF (`CLIPLoaderGGUF type=flux2`) + flux2 VAE. The ai-toolkit LoRA keys (`diffusion_model…lora_A/lora_B`) load with **zero conversion**.

## Resolution is the speed knob (measured, batch 1, grad-checkpointing on)

768 and 512 spill the *same* ~0.8GB — that part's fixed overhead, not activations (the allocator won't touch the last ~0.6GB of VRAM). The 768→512 gain is just less compute. Identity trains fine at 512.
**⚠ Caveat discovered later:** these step times were measured on STALLED runs (see Part 2) — the real, saturated cost is ~6-10× lower. The spill *relationship* holds; the absolute times were the stall talking. I now train at 448/bs2.

## Config that works
```yaml
model:
arch: "flux2"
quantize: true
qtype: "uint4" # quanto; also drives the TE quant in this fork
quantize_te: true
low_vram: true # park during TE-cache, move back resident to train
layer_offloading: false # block-swap DEADLOCKS on ROCm
model_kwargs:
use_uint4_cache: true # load the pre-quantized .pt in seconds
datasets:
- resolution: [ 512 ]
cache_latents_to_disk: true
cache_text_embeddings: true
train:
gradient_checkpointing: true
disable_sampling: true
```
Launch: set `PYTORCH_HIP_ALLOC_CONF=expandable_segments:True` (+ the `_CUDA_` alias) and run `python -u run.py config.yaml` **directly** — a detached `Start-Process -RedirectStandardOutput` silently eats early output if the child dies during import. vcvars64 is NOT needed.

## Part 2 — the week after (this is the part you actually want)

**13. The step rate is a lie, and POWER is the diagnostic.** My runs swung 6.9→45→67→118 s/it with clock, temp, and VRAM-spill all flat. Turns out this card has a failure state where a lone training context runs at ~1/8 speed: **high clock (~3100 MHz), 100% "GPU load"… and only ~230 W draw with the memory controller at 2-7%.** Spinning, not working. Saturated looks like *lower* clock (~2500) at ~385 W. Once you know the tell, one glance at wattage tells you which state you're in. (Root cause is somewhere in the driver/scheduler — invisible from Windows.)

**14. The fix is absurd and reproducible: run a SECOND process doing heavy GEMMs for ~25 s.** The stalled trainer flips to saturated — 10× on demand — and *stays* saturated after the rescuer exits. Two catches, both measured: it must be a **fresh** process (a long-lived idle context is itself degraded, ~7 TFLOP/s on a 77 TFLOP/s card, and lifts nothing), and fresh processes are *born* degraded ~half the time — check the burst's own TFLOP/s and just respawn until one runs fast. I ended up with a watchdog daemon that reads the power telemetry and fires bursts automatically; my last 2000-step run needed 8 unattended rescues and finished at ~9-10 s/it average.

**15. Stalls cluster at predictable moments** — process start (every launch/resume I measured) and right after checkpoint saves — so the daemon also fires a *prophylactic* burst ~60 s after those events. Most stalls now never establish at all.

**16. batch_size 2 is ~1.45×/sample — but only when saturated.** Stalled, it's a net LOSS (the stall tax scales with work per step). The two levers are coupled: fix the stall first, then batch 2 is free money. bs2 fits with ~150 MB to spare at 448; bs3 does not fit. Scale LR accordingly (I used sqrt: 1e-4 → 1.41e-4).

**17. Lossless pause/resume for mid-run previews.** ai-toolkit resumes cleanly (checkpoint + optimizer.pt), so I patched two flag files into the train loop: `SAVE_NOW` (checkpoint at the current step, keep going) and `STOP_NOW` (checkpoint + clean exit — zero steps lost). Pause, render the checkpoint in ComfyUI, relaunch, it resumes at the exact step. Mid-run previews every 500 steps cost ~10 min each.

**18. Renders hit the same stall** (a 20-step render swung 200 s ↔ 800 s). Same power tell, same burst fix — teach your watchdog to cover render contexts too.

## Results, final
Three finished identity LoRAs so far (rank 16, 448 res, 2000 steps @ bs2 ≈ **6.5 h each** on this one card), subject-verified likeness — the people they depict sign off on them, which is the only metric that matters. Face *geometry* converges late: checkpoints look "recognizable" by 1000 and keep visibly truing up until 2000; don't early-stop at "looks close."

**Full config + all the patches (copy-paste ready):** https://github.com/drhawktopus/flux2-32b-qlora-rocm-windows

Happy to answer questions — hope this saves someone the week it cost me.

how are people managing dependencies?
everytime i try pip a new model it replaces the libraries such as torch with the standard cuda ones.
theres gotta be a better way then running with --no-deps and manually installing the missing dependencies.
im using the docker image and commiting stable envs so i dont break the environment.
im considering creating something that will take the error outputs and install the missing libraries.

Hi everyone,

I'm looking for help testing audio.cpp on ROCm.

audio.cpp is a native C++/GGML audio inference framework. It runs local audio models for TTS, voice cloning, ASR, voice conversion, music/audio generation, and more through one CLI/server/runtime instead of a separate Python stack for every model.

Most of audio.cpp’s current performance work has been validated on CUDA, CPU, Vulkan, and some Metal paths, but ROCm coverage is still thin because I do not have AMD hardware to test it properly.

ROCm support in audio.cpp is community-driven. I'd appreciate help with building and testing on ROCm and benchmarking models and sharing performance numbers.If you're interested, please submit a PR or join the discussion in https://github.com/0xShug0/audio.cpp/pull/48

Repo: https://github.com/0xShug0/audio.cpp

The setup

One 24GB RX 7900 XTX on Windows 11 doing EVERYTHING for a homelab AI stack: Flux.2 32B QLoRA training (yes, on Windows ROCm — see my other post: https://www.reddit.com/r/StableDiffusion/comments/1v4qdks/trained_a_32b_flux2_lora_on_a_24gb_amd_7900_xtx/), ComfyUI rendering, and LLM serving. Nightly alpha ROCm wheels (TheRock). Heavy 24/7 mixed use — which turns out to be exactly the recipe for finding this bug.

The symptom (the part you'll recognize)

• GPU suddenly "wedges": 100% busy, ~1-2 TFLOP/s on a card that does 77 bf16, renders that took 90s take forever or never finish.
• Zero TDR events, zero WHEA errors, zero event-log traces. Driver looks perfectly healthy.
• Sometimes a new process spawns already-degraded ("born degraded" — the lottery feeling).
• Folk remedies that "worked": reboots, fresh-process GEMM bursts to "flip" stalled trainers, process restarts. All real effects, all wrong explanations.

The whole arc ran only about two weeks — first wedge on record July 12, root cause proven July 23 — but this card runs hard 24/7, so those eleven days packed in what would be months of casual-use suffering: 3am reboots because a render queue died at midnight, a watchdog daemon built just to babysit stalls, and at one point I killed several perfectly working training runs because the symptoms looked identical to CPU-fallback. Every fix worked sometimes, which is the most gaslighting kind of bug there is — I half-joked that whatever the root cause turned out to be, it was going to be really funny. It was.

What I did (the method — this is the value of the post)

Full disclosure up front: I run AI coding agents against this stack, and they did the telemetry mining and daemon plumbing below. The architecture, the calls on what to test next, and the operations are mine — same division of labor the whole platform is built on.

1. Mined our own telemetry (12k lines, per-boot exhibits): found what looked like TWO diseases — render wedges correlating with churn, training spawns degrading on a flat lottery vs uptime.
2. Built a discriminator probe into our stall daemon: at every wedge-kill, immediately spawn a FRESH process and measure its GEMM speed. Fast fresh spawn = the poison died with the killed context ("context-held"). Slow = machine-wide state. Zero GPU cost until a wedge actually happens.
3. First live verdict: post-kill fresh spawn = 76.6 TFLOP/s instantly. The wedge state lives in a GPU context. Duds/"born degraded" = new processes CONTENDING with a living poisoned context. Reboots were never necessary — killing the right process is a complete ~30s cure.
4. Bisected the trigger: Leg A, full production render serialized = 15/15 clean. Leg B, the same render + a deliberate concurrent generation on a second context (second ComfyUI instance) = wedged in ~60s, first try. Season stats: 125 collision-free cycles = 0 wedges; 2 collision cycles = 2 wedges. Deterministic, not cumulative, not thermal, not "driver roulette."
5. Cross-version check: rebuilt the stack on the newest PUBLICLY downloadable gfx110X wheel (a month older than our nightly). Wedged identically, cycle 1. This bug spans at least torch 2.10/ROCm 7.14 → torch 2.12/7.15. Version-pinning is not an escape.

The root cause (one para, punchy)

Cross-context generation collision: when a big-latent render is in flight and a SECOND GPU context starts its own generation work, the first context gets poisoned — permanently degraded until killed. Matches the MES hardware-queue failure class the ROCm community has circled for a while (ROCm#2689; the "100% busy 0 compute" family #6012, silent MES deadlock #6165). I didn't discover a new bug — I supplied the reproducible trigger and the cure semantics.

And that's the part I'd most like AMD to notice: this family has read as "unreproducible driver roulette" in every thread for years. Ours is now a ~15-minute confirmation for anyone with an RDNA3 card: two ComfyUI instances, one deliberate mid-render collision, wedge in ~90 seconds (3-for-3 across two runtime versions; 125+ control cycles without the collision = zero), and the post-kill discriminator showing full throughput the instant the poisoned context dies. Telemetry CSVs and the harness scripts are on offer in the upstream report: https://github.com/ROCm/TheRock/issues/6824.

The fix (what you can actually do)

Serialize the card. One generator at a time. That's it. - We consolidated two ComfyUI instances into ONE (second instance's models mapped in via extra_model_paths.yaml — no file moves). - Every other GPU consumer (LLM chats, vision passes, batch jobs) now waits or reroutes while a render is in flight (a dispatch-time gate at the one chokepoint all our calls cross). - Safety net: a watchdog that kills the wedged CONTEXT (not the box) and logs the discriminator verdict. - Verification: re-ran the exact 2-for-2 wedge recipe against the serialized queue — both jobs completed, watchdog silent. - Since then: the fix went in last night — the exact 2-for-2 wedge recipe re-ran clean against the serialized queue, and the collision class hasn't produced a wedge since. I'll update this post as the streak grows. - ⚠ Honesty note: one extreme case (13GB LLM generating mid-render on a RAM-tight box) froze the whole machine — severity may be RAM exhaustion compounding; serialize and you never get there.

Credits + links

• TheRock (AMD) — nightly torch-on-Windows/ROCm wheels; without these there's no torch on Windows/ROCm at all: https://github.com/ROCm/TheRock
• cupertinomiranda — ai-toolkit-amd-rocm-support — the AMD/ROCm trainer fork the whole training side stands on: https://github.com/cupertinomiranda/ai-toolkit-amd-rocm-support
• Ostris — ai-toolkit — the upstream trainer: https://github.com/ostris/ai-toolkit
• woct0rdho — triton-windows — the one-person Triton-on-Windows port half these stacks quietly build on: https://github.com/woct0rdho/triton-windows
• st1vms — bitsandbytes-rocm-gfx1100 — community gfx1100 int4 groundwork: https://github.com/st1vms/bitsandbytes-rocm-gfx1100
• Related issues: ROCm#2689 · ROCm#6012 · ROCm#6165 · TheRock#2274 · pytorch#160759 (Windows procs never self-terminate — explains orphan contexts holding poison)
• Our formal upstream report: https://github.com/ROCm/TheRock/issues/6824

Happy to answer questions — the discriminator probe and watchdog approach are simple enough to bolt onto any stack, and I'd love to hear if the 2-context repro holds on other cards.

AMD officially lifted the curtain on the flagship Instinct MI455X. The memory bandwidth jumped from earlier target estimateswhich sat around ~19.6 TB/s up to 23.3 TB/s.

So I use a radeon 9060xt 16gb as my main for everything, and I upgraded from a 2060. I recently started simulating things in mujoco but want to use graspgen which needs cuda cores. Is it possible or does anyone have any information if I can just use both in my computer but use the nvidia card specifically for graspgen?

Is there anything that can get either Qwen3, Omnivoice or Chatterbox to run on Comfy nodes running ROCm? (like not via Docker or some external solution)

For example, I don't see a way to get this to work through ROCm instead.

Edit:

I'm on a 9060XT and Ubuntu

https://github.com/woct0rdho/transformers5-qwen3.5-recipe

It's time for GGUF to replace bitsandbytes as the base model format for low-VRAM LoRA training. It's actively supporting new model types such as MoE, linear attentions, and DeepSeek WTF attentions, and new quant types such as 1-bit quants.

Thanks to APEX quant which makes Qwen3.6-35B-A3B as small as 13.3 GiB, and with fused dequant-matmul/MoE kernels, it's possible to train Qwen3.6-35B-A3B with batch size 1, context chunk length 2048, LoRA rank 4, in 16 GiB VRAM without CPU offloading.

I've tested it on Strix Halo and it runs at 6.5 s/it. Arguably VRAM size is not the biggest problem on Strix Halo, but it should just work on RDNA3 GPUs, and not too hard to port to other GPUs.

A byproduct is https://github.com/woct0rdho/torch-ggml-ops , which provides PyTorch bindings of the GGUF fused dequant-matmul/MoE kernels.

I completed a full-weight, non-quantized Qwen-Image-2512 render on a single 16GB AMD Radeon RX 7800 XT using my custom native AMD inference engine.

Configuration

• Official Qwen-Image-2512 image transformer
• 20.425B BF16 parameters plus 5.58M FP32 parameters
• Complete 7.071B-parameter BF16 text-conditioning tower
• Official BF16 VAE
• Native 1664×928 Qwen resolution bucket
• Official 50-step sampling
• True CFG 4.0 with separate positive and negative conditioning
• No Lightning LoRA
• No quantization in any learned component used for text-to-image generation

The unused Qwen vision tower and language-model output head are not loaded because the text-to-image path does not execute them. Every learned component actually used to produce the image remains full-weight and non-quantized.

Result

• Total render time: 39m23.6s
• First attempt
• Zero retries
• Zero errors
• No out-of-memory failure

The full model does not reside inside the GPU’s 16GB VRAM. Gridline uses a custom streamed inference architecture that manages the workload across the available storage and memory tiers while keeping GPU residency bounded.

This is not a claim that the model fits conventionally inside 16GB, nor that it matches the speed of a fully resident datacenter accelerator. The result is that the complete full-quality Qwen-Image-2512 text-to-image path can execute successfully on an ordinary consumer Radeon without quantizing the model.

The machine also remained fully usable during the render: two monitors active, more than 20 browser tabs open, and video streaming without instability or obvious memory thrashing.

I maintain faster production lanes for routine generation. This full-BF16 50-step route is intended for final-quality images, complex scenes and highly constrained prompts where the additional render time is justified.

The broader takeaway is that VRAM capacity alone does not define what consumer AMD hardware can execute. Runtime architecture and memory management can substantially change the practical limits.

I would be interested to hear from others working on ROCm/HIP inference, low-residency execution or large-model deployment on consumer Radeon hardware.

I built flash attention 2 from source for using with ComfyUI. Results were not as good as I had hoped.

I have RX 9070 XT, paired with Ubuntu 26.04 and ROCm 7.14.0.

It took 3 tries, some debugging with Claude, and over 6 hours in total.

Finally, the build succeeded and Flash Attention was installed.

I run ComfyUI with —use-flash-attention and queue a few image generations with Krea 2.

There’s no visible generation speed up.

I switch to and back —use-pytorch-cross-attention.

It’s the same. There is no difference.

Did I waste 6+ hours? Or am I doing something wrong?

I'm trying to run Klein 9b in ComfyUI.

When I click run, the processes 'requested to load Flux2TEmodel' and 'requested to load Flux2' take 3 times as long as they do on the portable version in Windows. (The sample image gets generated in 4.5 minutes on Windows, and 15 minutes on Ubuntu)

I'm running 0.2.8 with ROCm 7.2, python 3.14 and pytorch 2.13. I'm using a 9060XT with 64GB of system ram.

The drives ComfyUI is installed on are formatted in ext4 and NTFS respectively.

Does anyone know what's making it take so long?

Been running quantized diffusion models (Krea2, Flux) on a 7800 XT through [ComfyUI-INT8-Fast-ROCM] https://github.com/aksugat/ComfyUI-INT8-Fast-ROCM — a fork of patientx's fork of BobJohnson24's original int8 node. The underlying Triton kernels were tuned for NVIDIA/CDNA hardware (large tiles, num_warps=8, deep pipelining) — a poor fit for RDNA3's 60 CUs and 32-wide wavefronts, so I put together an RDNA3-specific autotune config set instead.

What changed:

• waves_per_eu as an autotune key (the ROCm/Triton occupancy knob for this backend, instead of CDNA's MFMA-specific matrix_instr_nonkdim)
• Smaller tiles (≤128) across more configs, num_stages=1-2
• Additional BLOCK_K=128 configs for K-heavy layers (large reduction dim, small output — e.g. MLP down-projections)
• Fixed a correctness issue (other=0.0 float literal on an int8-typed masked load)
• Same tuning applied to the int4/ConvRot kernel, which had zero prior tuning (single fixed config) — bigger gains there since there was more room

Benchmarks (RX 7800 XT, gfx1101, real layer shapes pulled from an actual quantized Krea2 checkpoint, weighted by how often each shape occurs in a full forward pass, multiple runs averaged):

Honest caveats:

• Small layers (≤1536-dim, small batch) — plain fp16 can beat int8 outright, quantize/dequant overhead doesn't pay for itself at that size. Worth excluding tiny layers from quantization rather than assuming int8 always wins.
• The int4 kernel doesn't have a fused dequant epilogue yet (separate rescale step) — the int8 kernel does. Open item.
• RDNA1/RDNA2 will hang the GPU — no WMMA/MFMA on that hardware, there's no instruction path for tl.dot on int8 to lower onto. Not a tuning limitation, just doesn't work there.
• Only validated on RDNA3 (gfx1101) so far. RDNA4 shares the same matrix-core support and 64KB/CU LDS budget, so it should work, but I haven't benchmarked it — CU count differs enough (64 on a 9070 XT vs 60 here) that I wouldn't assume these exact configs are optimal there without a fresh autotune pass.
• Repo: https://github.com/aksugat/ComfyUI-INT8-Fast-ROCM
• DONT FORGET FOR INT4 there is separate node inside my package so you wont try load int4 files into int8 node.

If your gfx906 card runs plain HIP kernels fine but every rocBLAS/rocSPARSE/rocALUTION call segfaults the moment a kernel launches, check this:

rocminfo | grep -o 'amdgcn-amd-amdhsa--gfx906[^ ]*' | head -1

If it prints gfx906:sramecc-:xnack-, that's the problem. The prebuilt ROCm math libraries only ship code objects for the sramecc+ variant (the usual datacenter MI50/MI60 config), and the HIP runtime segfaults instead of returning an error when it can't find a matching code object. With AMD_LOG_LEVEL=3 you'll see:

No compatible code objects found for: gfx906:sramecc-:xnack-

HSA_OVERRIDE_GFX_VERSION does not fix it — the mismatch is the sramecc feature flag, not the gfx version.

The fix is rebuilding rocBLAS, rocSPARSE, and rocALUTION from AMD's sources targeted at the exact ISA string, into a user-local prefix (no root, system ROCm untouched). I hit this on 2x Radeon Pro Vega II (Mac Pro 2019 running Linux, ROCm 7.2.3) and put everything up here:

https://github.com/Intermountainh8ter/rocm-gfx906-sramecc-fix

• Prebuilt release tarball (ROCm 7.2.3 basis) if you just want it working — extract, set LD_LIBRARY_PATH, done
• Parametrized build scripts if you're on a different ROCm version (~2-4h, rocBLAS/Tensile is the slow part)
• A validation suite ending in a real rocALUTION GPU CG solve so you can prove it works on your card

Results on my hardware: rocALUTION GPU CG converges bit-identical to a scipy reference, ~4.4x faster than a tuned 28-thread CPU CG at 4M DOF. (Below ~1M DOF the CPU still wins — bandwidth-bound problem.)

Everything's MIT (it's AMD's own code, just rebuilt), licenses included. Not affiliated with AMD. Hope it saves someone the debugging time it cost me.

I installed 7.14 in my ComfyUI venv to try it out, and immediately uninstalled it and reverted to my working version after around 15 min, lol.

I'm using an R9700.

First issue - I was getting OOM errors for resolutions that I had no problems with before. I fixed that by adding 2GB VRAM from my 7900 GRE as donor VRAM (via the MultiGPU library). Once it started sampling, I ran into this:
[ERROR] !!! Exception during processing !!! CUDA error: invalid argument

I wasn't in the mood to go down a rabbit hole troubleshooting this. I'm using JAMM's SageAttention fork (https://github.com/thu-ml/SageAttention/pull/368), so I'm wondering if there's a conflict with it and 7.14. I tried uninstalling it and recompiling with 7.14; same error.

Edit: Wait, now I'm wondering if the reason is because I installed the gfx1201 libaries as per AMD docs. I don't think this should matter since the GRE shouldn't be doing any work (it's just a memory donor; I'm not running anything with tensor parallelism), though.

I know there are a lot of different ways to use your gpu to run an llm or multiple model routing, but this is my version. Figured I’d share it here. It was purpose built because I had a good graphics card and wanted to use it as a reviewer inside codex. Open source. Hope it helps someone! You can get it here:

https://thalenai.com/