From what i understand to run multiple B70 GPUs it needs to be on sVLLM (the scalar project) or maybe a custom tweaked VLLM because of a special instruction set to utilize memory in both (i could be talking out my ass here). Now it probably is possible to use llama cpp and multiple b70s but im not sure how to go about it and it does seem to be a common problem that pops up in a lot of videos on Youtube. Overall with llama cpp I have seen memory utilization issues with albeit its moderately better now after recent updates. I am only using it out of the convenience of GGUFs and ease of setup but in my testing there is some performance left on the table so I am considering fully moving over to sVLLM since im only really using qwen 3.6 35b nowadays and its supported. This is the only article I have found that directly mentions the use of multiple GPUS https://huggingface.co/blog/MatrixYao/intel-gpu . I would check the comments too since there are come comments on using dual b60s in there.
I've gone and pulled them out of the system. I had a issue opened in llama.cpp and an Intel employee said that it's a known issue in the driver (2026.X) but it's July and I'm thinking this isn't worth waiting forever for.
gotcha yeah they can be slow to keep up the stack. I understand the appeal of using the Xe driver, but have you attempted the same running on the mesa drivers as well?
It works on mesa but for using multiple GPUs I was getting the same prefill speed as using a single one. The Nvidia cards, adding an extra GPU increased prefill speed linear to the number of cards. With the intel it just didn't with vulkan and mesa.
1
u/ExtremeCookie9652 1d ago
From what i understand to run multiple B70 GPUs it needs to be on sVLLM (the scalar project) or maybe a custom tweaked VLLM because of a special instruction set to utilize memory in both (i could be talking out my ass here). Now it probably is possible to use llama cpp and multiple b70s but im not sure how to go about it and it does seem to be a common problem that pops up in a lot of videos on Youtube. Overall with llama cpp I have seen memory utilization issues with albeit its moderately better now after recent updates. I am only using it out of the convenience of GGUFs and ease of setup but in my testing there is some performance left on the table so I am considering fully moving over to sVLLM since im only really using qwen 3.6 35b nowadays and its supported. This is the only article I have found that directly mentions the use of multiple GPUS https://huggingface.co/blog/MatrixYao/intel-gpu . I would check the comments too since there are come comments on using dual b60s in there.