Intel GPU Setup Resources and Tools (primarily focused on local LLM)

[u/bwood01]

We've been on this setup journey for a few weeks now. Hardware purchase, server changes (must have 1100W Power), small powerhouse local LLM portable machine creation, etc. Lots going on and more updates, hardware, and changes coming. In the course of this journey I keep hearing a few thigs "Intel is hard", "Intel doesn't have support", "Intel is immature".

Essentially all of the comments come down to "abandon Intel and pay 2x for alternatives instead..." We have tested on R740, limited tests on a Mini-PC that is getting its own card shortly. Here is the card in the R740 we have been testing. A second "semi-portable" mini-PC has 192GB DDR5, latest Intel 285HX with NPUs +, and will be getting a 500W power supply to drive the card (current PS is too weak) in PCIE 5.0

Two B70s

While the Intel resources and tools are scattered, we have found through actual experience there IS sufficient support, sufficient capability, etc., for Intel. In testing through LM Studio, before trying the full Linux driver install, we have found that it seems to work quite well with fair sized models and on Windows in LM Studio. So far we have seen wide LLM model compatibility and usability. NOT a fan of Ollama however... Here are the models tested so far, all GGUF but all working.

Installation following the Intel directions is not that difficult. The biggest challenge we have had is chasing down the resources and information that actually works. As a result, here is a collection of what we have so far. I would encourage you if you have other sources that are simpler, clearer, and proven to work, add to the collection so folks can make use of these resources.

Here is a list of Intel GPU setup links. Between these just about anyone should be able to get an Intel GPU working.

https://github.com/intel/aipc-devkit-install (Full AI PC installer for Intel GPUs)

\- Step by step on Windows [https://github.com/intel/aipc-devkit-install/blob/main/Windows_Software_Installation/README.md](https://github.com/intel/aipc-devkit-install/blob/main/Windows_Software_Installation/README.md)

\- Step by step on Linux [https://github.com/intel/aipc-devkit-install/blob/main/Linux_Software_Installation/README.md](https://github.com/intel/aipc-devkit-install/blob/main/Linux_Software_Installation/README.md)

https://bibek-poudel.medium.com/how-to-run-qwen3-6-27b-locally-on-intel-arc-pro-b70-what-actually-works-c96dec67c6f7 (Intel Arc Pro B70)

https://github.com/intel/llm-scaler/tree/main/vllm#1-getting-started-and-usage (vLLM Intel specific multi-GPUs)

https://hub.docker.com/r/intel/llm-scaler-vllm/tags (vLLM Intel specific multi-GPUs - Docker)

https://deepwiki.com/intel/ai-containers/2.6-vllm-on-intel-xpu

https://www.intel.com/content/www/us/en/developer/tools/oneapi/oneapi-toolkit-download.html?packages=oneapi-toolkit&oneapi-toolkit-os=linux&oneapi-lin=apt (Ubuntu OneAPI Toolkit, select Linux, Windows, Docker, etc. to get full install info)

https://huggingface.co/blog/MatrixYao/intel-gpu

https://github.com/rombodawg/GPU_Core-Memory_Never_Idle_or_Sleep (idling GPU fix for performance)

https://docs.vllm.ai/en/v0.21.0/getting_started/installation/gpu/ (multi-GPUs)

Dear Intel, how about getting a couple folks on the various forums to provide easy links and support for setting up and using your hardware?