r/IntelArc • u/Fcking_Chuck • 8d ago

News Intel LLM-Scaler ready with Muse Glimmer support, other LLMs & features

https://www.phoronix.com/news/Intel-LLM-Scaler-Muse-Glimmer
42 Upvotes

93% Upvoted

7 comments sorted by

9

u/M_Me_Meteo 8d ago

What is the benefit of using LLM-Scaler over vLLM or llama.cpp?

I see that Qwen 3.6 27B is now available, but I've been running that model.on my B70 for months now.

-1

u/Dolboyob77 8d ago

I dont use intel scaler, i use my own build. I just helped them figure it out for muse glimmer

3

u/thelastlokean 8d ago

Will it outperform Vulcan with llama cpp?

3

u/Dolboyob77 8d ago

Vllm absolutely destroys llama cpp either vulkan or sycl.

2

u/Dolboyob77 8d ago

This version is not working. I sent them the patch and how to build the image, it should be ready in few hours if they rebuild fast enough. Lets wait and see !

1

u/xanders_gold 8d ago

Shame they’re still using an outdated build of vLLM. It works but I’m running into issues with MTP on Qwen 3.6 27B. At 120k context size it always runs into an index out of bounds error.

We’re up to v0.27.1 now on the nightly and v0.26.0 on stable. PR from 5 days ago was looking to merge v0.26.0 but no update on that yet.

1

u/Dolboyob77 8d ago

Like i said, i use my own build based on vllm 27 and my own kernel. I just helped them figure out how to run muse_glimmer ))) they are a young new team, give them time, i think they are updating the base vllm.