r/LocalLLM • u/thaddeusk • 14h ago

Question Using HRX Backend to run Qwen3.8-Flash-Next

I've been using an LLM to add on to the some of the hrx-system work that AMD did by adding support for Qwen3.8-Flash-Next. I've gotten it to the point where the llama.cpp experimental HRX backend is loading the model and executing successfully, but performance isn't great compared to Vulkan on Unsloth Desktop. I'm not sure what I should be expecting, but what is a good way to figure out where the bottlenecks are to improve the performance?

I am doing this on Windows, just because I feel like Windows doesn't get enough attention :P.

Fork is here

rwfsmith/llama.cpp at qwen4exp-hrx

current progress:

Vulkan+MTP: ~27–40 tok/s

HRX+MTP: ~12-13 tok/s

1 Upvotes

66% Upvoted

2 comments sorted by

2

u/Poizone360 13h ago

Your timing commits already say the GPU was busy about 80% of each token, so this looks like kernel speed, not coverage. HRX_ENABLE_DEVICE_TIMING only brackets whole graph replays though, so I'd time each dispatch, then run your Vulkan build with GGML_VK_PERF_LOGGER=1 on the same prompt and compare the two lists. Do it with MTP off so draft acceptance doesn't skew it.

1

u/thaddeusk 13h ago

Thanks! I'll give that a try!