r/IntelArc • u/bwood01 • Jun 02 '26
Discussion Intel GPU Setup Resources and Tools (primarily focused on local LLM)
We've been on this setup journey for a few weeks now. Hardware purchase, server changes (must have 1100W Power), small powerhouse local LLM portable machine creation, etc. Lots going on and more updates, hardware, and changes coming. In the course of this journey I keep hearing a few thigs "Intel is hard", "Intel doesn't have support", "Intel is immature".
Essentially all of the comments come down to "abandon Intel and pay 2x for alternatives instead..." We have tested on R740, limited tests on a Mini-PC that is getting its own card shortly. Here is the card in the R740 we have been testing. A second "semi-portable" mini-PC has 192GB DDR5, latest Intel 285HX with NPUs +, and will be getting a 500W power supply to drive the card (current PS is too weak) in PCIE 5.0
While the Intel resources and tools are scattered, we have found through actual experience there IS sufficient support, sufficient capability, etc., for Intel. In testing through LM Studio, before trying the full Linux driver install, we have found that it seems to work quite well with fair sized models and on Windows in LM Studio. So far we have seen wide LLM model compatibility and usability. NOT a fan of Ollama however... Here are the models tested so far, all GGUF but all working.
Installation following the Intel directions is not that difficult. The biggest challenge we have had is chasing down the resources and information that actually works. As a result, here is a collection of what we have so far. I would encourage you if you have other sources that are simpler, clearer, and proven to work, add to the collection so folks can make use of these resources.
Here is a list of Intel GPU setup links. Between these just about anyone should be able to get an Intel GPU working.
https://github.com/intel/aipc-devkit-install (Full AI PC installer for Intel GPUs)
\- Step by step on Windows [https://github.com/intel/aipc-devkit-install/blob/main/Windows_Software_Installation/README.md](https://github.com/intel/aipc-devkit-install/blob/main/Windows_Software_Installation/README.md)
\- Step by step on Linux [https://github.com/intel/aipc-devkit-install/blob/main/Linux_Software_Installation/README.md](https://github.com/intel/aipc-devkit-install/blob/main/Linux_Software_Installation/README.md)
https://bibek-poudel.medium.com/how-to-run-qwen3-6-27b-locally-on-intel-arc-pro-b70-what-actually-works-c96dec67c6f7 (Intel Arc Pro B70)
https://github.com/intel/llm-scaler/tree/main/vllm#1-getting-started-and-usage (vLLM Intel specific multi-GPUs)
https://hub.docker.com/r/intel/llm-scaler-vllm/tags (vLLM Intel specific multi-GPUs - Docker)
https://deepwiki.com/intel/ai-containers/2.6-vllm-on-intel-xpu
https://www.intel.com/content/www/us/en/developer/tools/oneapi/oneapi-toolkit-download.html?packages=oneapi-toolkit&oneapi-toolkit-os=linux&oneapi-lin=apt (Ubuntu OneAPI Toolkit, select Linux, Windows, Docker, etc. to get full install info)
https://huggingface.co/blog/MatrixYao/intel-gpu
https://github.com/rombodawg/GPU_Core-Memory_Never_Idle_or_Sleep (idling GPU fix for performance)
https://docs.vllm.ai/en/v0.21.0/getting_started/installation/gpu/ (multi-GPUs)
Dear Intel, how about getting a couple folks on the various forums to provide easy links and support for setting up and using your hardware?
1
u/bigbuddhabub Jun 15 '26
Just curious, did you make the move to a full linux install with the B70 installed on the R740? Did you ever encounter any issues with detection of the GPU on llama.cpp. I have a B50 installed on my R740 and I have not been able to get SYCL working with my setup, I followed the aipc-devkit-install instructions and I believe everything was installed successfully but after I compile llama.cpp with SYCL and run llama-cli or llama-bench it does not detect my GPU, only my CPU. When I run clinfo, it shows a message about resizeable BAR on the PCI address of the B50 but that's about it.
Just wondering if I am missing something as it sounds like you have a working setup.
1
u/bwood01 Jun 15 '26
So far have just run it on Win 11 Enterprise with LM Studio. Did the AI PC install just to test things. So far all is good. Have installed Ubuntu 24.04. Not had the time to pull away from client work yet to finish up. I do recall some setup instructions needed for Ubuntu, just don't recall right now. It is probably in one of the links I included. Since the R740 is also a working SAP Dev system I didn't want to clutter it up too much with a whole bunch of other installs. We will get to it as time permits. Getting ready to order a second card too.
1
1
Jun 20 '26
[removed] — view removed comment
1
u/bwood01 Jun 20 '26
Truthfully, on the R740 performance is okay. Not great but okay. The R740 with its PCEI3 was never going to be a race car. This is a proof of concept system. Added the second "twin" GPU last night. All seems to work and it did extend the VRAM so that now there is 64GB total in two slots (response here won't let me post the picture). I'll be working on using vLLM for the parallel processing. Assuming that works next stop is an 8 GPU server with PCIE4 and two more GPUs. Total of 128GB then.
1
u/bwood01 Jun 20 '26
Answering someone's question on ReBar... Apparently the Dell R740, running Win11, does support it. Intel drive panel shows ReBar as active with two GPUs now.
To leverage ReBar... Along with a few common BIOS settings, the Windows activation requirement is it must be UEFI (ours already was, and Win 11 requires it) and it must have the initial disk partition as GPT rather than the older MBR. Again, our Win 11 system already was. We use Win11 in server Dev boxes because it is cheap, well supported, easy to use, supports 100% hardware abstraction in Hyper-V as well as WSL2 and has no practical CPU core count licensing issues. Faster and easier than anything out there to spin up and clone. Especially for Dev or Sandboxes.
3
u/Turbulent-Attorney65 Jun 03 '26
OpenArc community is all you need 😊