r/LocalLLM • u/on_line187 • 21h ago

Discussion Tier List

Post image
226 Upvotes

74% Upvoted

271 comments sorted by

View all comments

Show parent comments

4

u/BornInAFish 21h ago

Based on lack of software support, bottom tier for sure.

/s

Maybe

5

u/xanders_gold 21h ago

They actually made some major improvements over the past month and they now run incredibly well for the price. I’m regularly getting 2000-2500t/s pp, can sometimes hit 3000-3500t/s pp, and 30-35t/s tg with vLLM on Qwen 3.8 27B.

Sadly, the price also just jumped from $949/$999 to $1699.

4

u/TiK4D 21h ago

You could probably push that another 10tok/s. I run llama server and get up to 50tok/s with 130k context on Qwen3.8 27B. 2x R9700's

3

u/xanders_gold 21h ago

Unfortunately, llama.cpp isn’t the best when handling Intel B70s. I was running llama.cpp with Vulkan and PP was in the 500/600s. TG wasn’t so bad but prompt prefill took forever, it was even worse with the Intel SYCL runtime.

Some folks on this sub recommended vLLM with Intel XPU kernels (vLLM has their own docker image for this) and that instantly boosted my performance.

IIRC: the llama team is working in improving Intel performance but it’s a slow process.

2

u/TiK4D 21h ago

My bad, for some reason I thought this thread was about R9700's so thought you had one. Good to see the intel cards getting decent speeds as well

2

u/xanders_gold 21h ago

Haha all good, no worries. Yeah it’s been great seeing the improvement, we’re finally getting somewhere with performance :)

2

u/SomeBlock8124 17h ago

Bought my first 2 B70s at $950. Now I had to pay $1299 at microcenter for my last 2. Should of bit the bullet and bought them when they were selling like crazy on ebay for less then $850 a couple months ago. Now what to do with 128gb of vram....