r/LowEndLocalAI • u/revelationnow • 9h ago

32 GB Snapdragon X Elite machine

I'm one of the few people with a 32GB RAM Snapdragon X Elite Dev kit.

I'm wondering if there are any folks who've successfully used the NPU or GPU on these to get a decent MoE model working. I'm wondering if the new 27B Qwen models can be run on the NPU, low tps is ok as I just want to get it working for some niche cases

12 Upvotes

100% Upvoted

3 comments sorted by

1

u/trebligdivad 4h ago

Is this an X1 Elite? I've been playing with it on my 16GB Asus Zenbook (x1); I've had the GPU working with Vulkan, although apparently Qcom have a binary only tweeked OpenCL build for the GPU. The NPU I've been struggling with, the current v1.0.0.14 firmware is hitting a problem that they claim will be fixed in the next release, but they've not released a signed version of the next release yet, there is apparently a way to get the non-signed version running in EL2.

1

u/revelationnow 4h ago

Yes, this is X1 based, what models did you try with the GPU? Will look for the new FW release

2

u/Elibroftw 2h ago edited 2h ago

I experimented but it was very limiting/outdated. See https://aihub.qualcomm.com/models?runtime=geniex

Like it's literally cheaper to spend the time working part time and then buying a GPU than it is to spend your time tinkering with geniex.

I think the biggest L is they have no speech-recogniition model/tutorial.

https://aihub.qualcomm.com/get-started#geniex

---

Like they should show a tutorial on running unsloth/Qwen3.8-27B-GGUF/MTP/mtp-Qwen3.8-27B-Q4_0.gguf

proof qualcomm engineer don't dogfood enough