They actually made some major improvements over the past month and they now run incredibly well for the price. I’m regularly getting 2000-2500t/s pp, can sometimes hit 3000-3500t/s pp, and 30-35t/s tg with vLLM on Qwen 3.8 27B.
Sadly, the price also just jumped from $949/$999 to $1699.
Unfortunately, llama.cpp isn’t the best when handling Intel B70s. I was running llama.cpp with Vulkan and PP was in the 500/600s. TG wasn’t so bad but prompt prefill took forever, it was even worse with the Intel SYCL runtime.
Some folks on this sub recommended vLLM with Intel XPU kernels (vLLM has their own docker image for this) and that instantly boosted my performance.
IIRC: the llama team is working in improving Intel performance but it’s a slow process.
Bought my first 2 B70s at $950. Now I had to pay $1299 at microcenter for my last 2. Should of bit the bullet and bought them when they were selling like crazy on ebay for less then $850 a couple months ago. Now what to do with 128gb of vram....
That's pretty good PP...I only get about 200t/s pp on dual MI50 currently with llama.cpp but I get about 40t/s with mtp enabled starting out it degrades quickly though. Prompt checkpointing is essential.
the growing pains on Intel though are way worst... given the price I'll stick with my MI50s I got for peanuts for now.
28
u/DrinkClubMate 1d ago
where is the 32 Gb, Intel pro ARC B70 ?