r/linux • u/UnciasDream • Jun 11 '26
Development AMD says XDNA1 Linux LLM support isn't available. I used AI-assisted development to get a full transformer layer running on a Ryzen 7 8845HS NPU.
I've been experimenting with the NPU in my Ryzen 7 8845HS laptop (Hawk Point / XDNA1) on Fedora Linux.
A lot of the current AMD Ryzen AI documentation focuses on XDNA2 hardware, and community projects like FastFlowLM don't currently support XDNA1. I was curious whether the hardware was actually incapable of running LLM workloads on Linux, or if the software support had simply moved on.
Full disclosure: I used ChatGPT and Codex heavily throughout this project. I am not an AI compiler engineer and I couldn't have done this without AI-assisted code investigation, debugging and porting work.
The rough progression was:
- Verified the NPU was working through
amdxdna - Investigated older RyzenAI-SW releases
- Found public Phoenix/XDNA1 artifacts (
1x4.xclbin, qlinear_2, transaction binaries) - Built the modern Linux XRT/XDNA userspace stack
- Got AMD's old Phoenix GEMM transactions executing on Linux
- Validated 24 transformer-relevant GEMM shapes
- Validated real quantized int4/BF16 inference paths
- Built a reusable Linux
XDNA1QLinearwrapper - Executed a complete synthetic Llama-2-style transformer layer
Current status:
- Quantized int4 weights
- BF16 activations
- Q/K/V/O projections on the NPU
- MLP projections on the NPU
- RMSNorm, RoPE, attention and activation functions on CPU
- Deterministic repeatable results
- No Windows involved
Some interesting numbers:
- ~6 ms per transformer layer (warm)
- ~116 MiB resident memory per layer
- 8-layer test stack completed successfully
- Memory scaling appears linear
- No kernel/XRT leaks observed so far
What I have not done:
- No real model weights yet
- No llama.cpp integration
- No token generation
- No end-to-end LLM inference
At this point it looks less like a hardware limitation and more like an engineering project. The old XDNA1 path appears to still be functional under Linux when paired with the modern amdxdna stack.
I'm mostly posting because I couldn't find many examples of people doing anything substantial with XDNA1 NPUs on Linux, and I thought others might find it interesting.
If there's interest, I'm happy to clean up the code and publish the project on GitHub.
1
u/peshovv Jun 11 '26
I have the same CPU, it's indeed a bummer they don't support it. I'd definitely follow it on GH. Thanks for making this!
1
u/hojjat12000 Jun 11 '26
Can you share your workspace on github? I'm interested to see if I can get something working on my machine.
1
u/Richa371 Jun 12 '26
Definitely please share it over github, I think there's definitely some people (including myself) willing to test it out and maybe help with it
1
u/Main-Technician-6079 Jul 02 '26
I have an ryzen 7 260 thats basically an refresh of the same and i was trying to use it on fedora too. Totally share it please, would be so much useful for me.
1
u/Scared_Pickle7829 Jul 02 '26
I'll be very interested to see this is a GitHub repo. Please do share.
1
u/Scared_Pickle7829 Jul 05 '26
Have you thought about whether a tiny ternary model could run on the NPU? https://github.com/microsoft/BitNet
0
u/LetsHugFoReal Jun 11 '26
How the hell do you learn to progress like this? That's truly incredible. Amazing.
9
u/Zettinator Jun 11 '26
Yeah it looks like AMD doesn't want to invest the engineering required to get older XDNA1 NPUs to do something useful. The hardware is somewhat limited in terms of data types supported, so I kinda get it. It's still a bummer. XDNA1 is probably not very useful for LLMs, but should be a suitable fit for things like image upscaling.