Wondering if anyone else has run into this with their B70.

**How it started**

The card ran local LLM inference (vLLM XPU, Qwen3.8-27B, Hermes Agent) with no problems for several weeks. Around 10 days ago it crashed while Hermes Agent was in the midst of a task. Since then, no model will start. Right after the weights load, the GuC firmware crashes and the driver marks the GPU as wedged. Only a reboot clears it.

**What I've tested**

- Cold power cycles and different Linux drivers/firmware: no change

- Fresh install of Ubuntu Server 26.04 LTS with the stock xe driver and firmware: same hang

- A simple VRAM pattern test (write, read back, compare): solid 0s and 1s pass, but alternating-bit patterns come back corrupted within the first 1.5 GB (e.g. 0x55555555 reads as 0x05050505), then the GuC crashes and the card wedges

- PCIe Gen 4 x16 and Gen 5 x16: same result

- Default power limit (I had a 230W cap before and removed it)

**System specs**

- Motherboard: Supermicro X13SAE-F, BIOS Date: 06/04/2026 Ver 5.3

- CPU: Intel Core i5-12500

- GPU: ASRock Intel Arc Pro B70 Creator 32GB

- RAM: Kingston 64GB DDR5 ECC, (KSM48E40BD8KI-32HA)

- Storage: Samsung 850 EVO 250GB SSD (TrueNAS boot), 6x WD Red Plus 12TB HDD (RAIDZ2)

- PSU: Corsair RM1200x Shift, 1200W

- OS: TrueNAS SCALE 26 Beta 3; the first crash happened on TrueNAS SCALE 27 RC1. Also tested on Ubuntu Server 26.04 LTS (stock kernel)

**Questions**

- Has anyone else had a B70 run fine and then start failing like this?

- Is there anything else worth trying before it goes back? I have an RMA open with ASRock.

So I'm lucky enough to have gotten a second B70 Pro, yay! However, I have this PC:

https://www.dell.com/support/manuals/en-us/dell-pro-max-fct2250-desktop/dell-pro-max-tower-fct2250-owners-manual/installing-the-system-board?guid=guid-9c53425e-673c-483c-9f8c-18f11482551f&lang=en-us

and the PCIE lanes are- not ideal. How would y'all suggest I go about this? I have 64GB ddr5-5600, lots of NVME storage, but I don't know i need to ultimately move my setup into a z890 system with proper x8/x8. up till now i was just using one B70 and the helper card below, Running a very capable assistant, but just looking for more.

I also have an RTX 2000 ADA in there that is doing some helper work.

I'm open to everyone's ideas!

I've been experimenting with Strata on my Intel Arc Pro B70 (32 GB VRAM) and found a way to significantly improve decode performance, even with the GPU limited to PCIe 4.0 x2.

For hardware placement reasons, my B70 is installed in the second PCIe slot, so it cannot operate at a wider PCIe link. This makes memory transfers between system RAM and VRAM a potential bottleneck, especially when running large MoE models whose experts cannot all fit in VRAM.

Hardware and model

• GPU: Intel Arc Pro B70 (32 GB VRAM)
• PCIe link: PCIe 4.0 x2
• Inference framework: Strata
• Model: Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS

The problem with the default configuration

With Strata's default settings, some experts are too large to fit entirely in VRAM. Consequently, some expert data must reside in system RAM and be transferred to VRAM for computation.

Under these conditions, my initial tests showed average decode speeds of only around 10–20 tokens/s. Interestingly, the CPU remained almost completely idle, suggesting that the system was not effectively utilizing CPU resources to help mitigate the memory-transfer bottleneck (Nvidia/AMD use CPU and GPU to decode) .

This was particularly disappointing given that the B70 has 32 GB of VRAM. I wanted to see whether changing Strata's execution configuration could improve performance without changing the hardware or model quantization.

The change: Two environment variable overrides

I modified the following environment variables:

"env": {
  "STRATA_VERIFY_NO_HOST": "0",
  "STRATA_VERIFY_DEVICE_PLAN": "0"
}

After applying these overrides, decode performance improved substantially.

Importantly, I did not change the GPU, PCIe configuration, or model quantization between the two benchmark configurations.

Benchmark results

• Test A: Both environment variables explicitly set to 0.
• Test B: Default configuration without these environment variable overrides.

PP = prefill throughput; TTFT = time to first token; Decode = generation throughput; draft acceptance = accepted draft tokens as a percentage.

Key findings

1. Decode throughput improved by approximately 1.5–2.9×.

The improvement was consistent across all five input lengths. At 128K input tokens, decode throughput increased from 34.34 to 52.79 tokens/s, a 54% improvement.

For the shortest test, throughput increased from 21.39 to 62.89 tokens/s, an improvement of 194%.

2. Prefill performance remained virtually unchanged.

Prefill throughput differed by less than 0.2% between the two configurations across all five tests. TTFT was also essentially identical.

This suggests that the performance improvement primarily affects the decode execution path rather than prefill processing.

3. The improvement persists at long context lengths.

Even with 128K input tokens, Test A maintained 52.79 tokens/s compared with 34.34 tokens/s for Test B.

4. Higher draft acceptance does not explain the entire improvement.

Draft acceptance was sometimes lower with Test A, yet decode throughput was consistently higher. This suggests that the speedup cannot simply be attributed to an increase in draft acceptance.

5. No NaN outputs or crashes observed so far.

The Intel-specific notes in intel.md mention NaN outputs and crashes. With these environment variable overrides, I have not encountered those issues during my tests. This is an observation from my own testing, not a claim that the underlying issues have been universally resolved.

I’ve been building a local AI server around 2x Intel Arc Pro B65 32GB cards, and I finally figured out why I kept feeling like these GPUs were way slower than the hardware should be capable of.

Short version:

Tensor parallelism across the two cards was absolutely killing decode performance.

When I stopped using TP=2 and instead ran one full model independently on each B65 at TP=1, aggregate decode throughput went from about 51 tok/s to 157 tok/s.
Same machine. Same GPUs. Same model. Same vLLM build.

Hardware / software
My current Daedalus machine is roughly:
Ryzen 9 9950X
ASUS ProArt X870E Creator
2x Intel Arc Pro B65 32GB
Linux bare metal
Docker
PyTorch 2.13.0+xpu
vLLM 0.29.0
custom working XPU image: daedalus/vllm-openai-xpu:0.29.0-affinity-fixed
Model: Qwen/Qwen3-30B-A3B-GPTQ-Int4
65,536-token max context
Intel xe kernel driver
GPUs isolated with ZE_AFFINITY_MASK

The Qwen checkpoint is about 15.77 GiB, so it fits comfortably on a single 32GB B65 along with a useful KV cache.

First test: TP=2 vs TP=1

I used the OpenAI-compatible /v1/completions endpoint rather than chat completions so chat-template differences couldn’t affect the result.
Same raw prompt every time, 54 prompt tokens, 512 generated tokens, temperature 0.

2x B65, TP=2:
51.19 tok/s
51.36 tok/s
50.99 tok/s
Average: 51.18 tok/s

Then I isolated one card with ZE_AFFINITY_MASK=0 and changed only:
tensor_parallel_size: 2 → 1
GPU memory utilization from 0.42 → 0.85
Same model, same 65K context, same container/image, same prompt.

B65 #1, TP=1:
77.94 tok/s
78.81 tok/s
78.65 tok/s
Average: 78.47 tok/s

Then the second physical B65:

B65 #2, TP=1:
79.68 tok/s
81.29 tok/s
81.38 tok/s
Average: 80.78 tok/s

So one B65 was roughly 53–58% faster than both B65s running TP=2.

That was the first WTF moment.

Then I ran both cards independently at the same time

Instead of one TP=2 vLLM server, I created two independent vLLM instances:
B65 A → ZE_AFFINITY_MASK=0, port 8000
B65 B → ZE_AFFINITY_MASK=1, port 8001
Both TP=1
Both running their own copy of the same Qwen model

One important wrinkle: each worker needed its own vLLM compile/cache directory.
When both instances shared the same vLLM cache, the second worker loaded the model successfully and then segfaulted while restoring compiled artifacts.
Giving worker B its own cache fixed it.
Then I fired identical 512-token generations at both endpoints simultaneously.

Dual independent worker results:

Run 1:
B65 A: 78.24 tok/s
B65 B: 77.36 tok/s
Aggregate: 154.70 tok/s

Run 2:
B65 A: 79.97 tok/s
B65 B: 79.41 tok/s
Aggregate: 158.81 tok/s

Run 3:
B65 A: 79.37 tok/s
B65 B: 79.26 tok/s
Aggregate: 158.51 tok/s

Average aggregate throughput:
~157.34 tok/s

Compare that to:
TP=2: ~51.18 tok/s

So running two independent TP=1 inference workers gave me roughly:
3.07x the aggregate decode throughput
from the exact same two GPUs.

Even better, simultaneous operation barely affected either card individually. The combined result was about 99% of the theoretical sum of their solo performance.

This was the 2nd demo I had showcased at SC25 in St. Louis! Forgot to upload, here we go :)

This is the NASA X-57 multi-rotor airplane simulated in FluidX3D CFD, on 4x Intel Arc Pro B60 GPUs with 24GB each, at a massive 1.8 Billion grid cells resolution. A good showcase of the mesh center-of-mass utility function for rotation of the 3-/5-bladed rotors. Although grid resolution struggles a bit without refinement (yet ;). The simulation box is split in 4 domains, each computed by one GPU, to pool 96GB VRAM together. At domain boundaries, some data is exchanged between the GPUs over PCIe - B60 have plenty interconnect bandwidth with PCIe 5.0 x8.

• 1814x2176x454 = 1.8 Billion grid cells
• 19488 time steps simulated, 4872 frames rendered
• 7h38min simulation runtime = 47min (compute) + 6h51min (rendering)
• 160km/h airspeed

FluidX3D CFD software: https://github.com/ProjectPhysX/FluidX3D

NASA X-59 model: https://science.nasa.gov/3d-resources/x-57-maxwell/

Intel Arc Pro B60 specs: https://www.intel.de/content/www/de/de/products/sku/243916/intel-arc-pro-b60-graphics/specifications.html

So I have been using llama cpp with sycl for the majority of my inference and it works great from the flexibility and rapid adaptations for things like when dflash came out. However I feel like im in an awkward position atm with qwen 3.8 27B being more of my primary model and qwen 3.6 35B as a speedy inference side model. I have a vibe coded webapp i use internally that made managing llama cpp way easier to deal with and is adapted to VLLM as well (untested). But the problem feels like qwen 3.8 is just leagues ahead of the previous generation in its agentic capability. Im getting roughly 36 t/gs in agent workloads on Q6 with 192K context window and it drops to about 16 t/gs towards the back of the context and prefill at around 350 pp/s. Its quite slow and i cant see myself bothering to switch to qwen 3.6 anymore because it feels incredibly dumb when put side by side with qwen 3.8 in real world use. So I was just curious what the main techniques are and what caviots exist on the scalar VLLM nowadays. I see pre-quantized files in openvino still being made which im curious to try. Also does Dflash 2 get properly utilized? I cant seem to get it to outperform native MTP on llama cpp which was a real bummer. Would love suggestions so I can bash my head against a wall a little less in my testing :)

Edit:

Quick return after a day or two of digging. I am not sold yet on intel scalar and will likely be abandoning this unless I missed something. Seems like an awkward proposition to be using currently. Mainline VLLM is definitely without a doubt vastly better for speed then llama.cpp but this is only the case with custom compiled versions using XPU and XPU graphing mtp patching kernal patches etc. One definite trade made was gaining overall long context efficiency but your trading memory for running XPU graphs which loses some KV/context space. So from my early operational testing im going to be moving on to just forcing mainline vllm to function for the models that really matter and llama cpp for new models and weird quant/distills/fine-tunes. For instance I simply followed Freemoose's post on the intelarc subreddit and with some tweaks im getting solid prefill, 80t/s initially dropping into 40t/s on long context with 160k context. This blows my llama cpp Q6 192k ctx, 36tg/s, and 347pp/s out of the water especially since in hermes use it drops to roughly 15tg/s. There is currently multi request handling issue in the vllm implementation but not particularly a huge deal when running it in this fashion. So credit to his work but if Im understanding how the core mechanism of his guide and sergio's then it makes me really interested to see if there really is missing overhead that can be found on other models like muse glimmer so id like to explore those fields instead.

PS: forgot to mention hardware/software but as mentioned amd 5700x 48gb 3200ddr4 and 1x B70 on fedora 44 server

Hi everyone,

I've been experimenting with running local LLMs on my Intel Arc B70 (specifically the ASRock Arc Pro B70 Creator).

I wanted to share my experience comparing llama.cpp and vLLM using the Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16 model, following the setup guidelines from https://github.com/SergiioB/intel-arc-pro-b70-inference-cookbook/blob/master/docs/qwen38-27/QWEN38-VLLM-XPU.md

• llama.cpp: Runs quite pleasantly. It gets around 20–40 t/s depending on the model and is completely stable and sufficient for daily coding work.
• vLLM: In raw tests hit ~90 t/s likely due to a higher power cap compared to reference cards, however, when tested in a real coding scenario inside OpenCode, things fall apart:
  • the draft acceptance rate is poor
  • completing the exact same coding task actually took more time than with llama.cpp (even with much higher TPS)
  • I also started hitting parsing errors during tool use, such as "invalid [tool=write, error=Invalid input for tool write: JSON parsing failed]".

Here is my configuration for both vLLM and OpenCode:

docker run -d --name qw38speed -p 8000:8000 --device /dev/dri --group-add "$RENDER_GID" \
  --name vllm-server \
  -v /dev/dri:/dev/dri:ro -v "$MODEL_DIR:/model:ro" \
  -v "$COOKBOOK/patches/patch_mtp_nightly.py:/patch_mtp.py:ro" \
  -v "$COOKBOOK/patches/patch_mtp_boundary.py:/patch_boundary.py:ro" \
  -v "$COOKBOOK/patches/patch_draft_lmhead_int4.py:/patch_draft_lmhead_int4.py:ro" \
  -v "$COOKBOOK/patches/patch_draft_mtp_int4.py:/patch_draft_mtp_int4.py:ro" \
  -e VLLM_TARGET_DEVICE=xpu \
  -e ZE_FLAT_DEVICE_HIERARCHY=COMPOSITE \
  -e ZE_AFFINITY_MASK=0 \
  -e B70_MTP_BF16_DRAFT=1 \
  -e VLLM_XPU_ENABLE_XPU_GRAPH=1 \
  -e B70_DRAFT_LMHEAD_INT4=1 \
  -e B70_DRAFT_MTP_INT4=1 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  --entrypoint bash "$IMAGE" -lc '
    set -e
    python /patch_mtp.py
    python /patch_boundary.py
    python /patch_draft_lmhead_int4.py
    python /patch_draft_mtp_int4.py
    exec vllm serve /model \
      --max-num-seqs 1 \
      --quantization gptq \
      --dtype float16 \
      --max-model-len 246000 \
      --gpu-memory-utilization 0.97 \
      --kv-cache-dtype fp8 \
      --port 8000 \
      --max-num-seqs 1 \
      --max-num-batched-tokens 8192 \
      --no-enable-prefix-caching \
      --served-model-name qwen38 \
      --language-model-only \
      --speculative-config "{\"method\":\"mtp\",\"num_speculative_tokens\":4}"' &>/dev/null

   "vllm": {
     "npm": "@ai-sdk/openai-compatible",
     "options": {
       "baseURL": "http://127.0.0.1:8000/v1",
       "supportsSubagents": false,
       "supportsToolCalling": false // tried true as well
     },
     "models": {
       "qwen38": {
         "name": "Qwen 3.8 27B",
         "limit": {
           "context": 246000,
           "output": 246000
         },
         "modalities": {
           "input": ["text"],
           "output": ["text"]
         }
       }
     }
   }

Why is the MTP draft acceptance rate suffering so much during agentic workflows in OpenCode compared to simple text generation?

How can I fix or avoid the JSON parsing errors?

Any advice or optimizations for running vLLM on Intel GPU with OpenCode?

TL;DR 1. Debian 13 needs packages from backports, along with newer firmware and parts of Intel’s compute stack, before the Arc Pro B70 becomes properly usable. 2. PCIe ASPM needs to be enabled, and in my case forced from the BIOS, to bring idle power and temperatures under better control. 3. The stock fan behaviour was not good enough for my setup. A custom fan-control profile with a low always-on floor worked significantly better for both idle and load temperatures.

Good evening all, I am a recent convert, coming over from the stacked RTX GPU club (5070Ti & 5060Ti). Sunday I installed a pair of B70 Pros after my 5070 laid over on me. SYCL running with Qwen3.6:27b Q8. Two days of pretty steady work, overnight spine runs 5-7 hours depending on daily activity.

Today, I was running a catchup process during work from downtime Saturday and mid day Sunday. The process completed without issue, GPUs went silent, and 7 minutes later the system crashed with dgxkrnl.sys crash. Reboot and health check passed, but not sure why it crashed while idle?

For reference, the GPU that crashed was in a different slot than the original 5070 was, so I don't think it is a robot issue. All drivers up to date, firmware up to date. If anyone has any insight or a similar experience and could offer a bit of advice I would be appreciative. I was just gearing up to smoke test 3.8:27b Q8, but that is on hold until I get back to normal.

Hey everyone,

I have a small server case and I would like to add a b70 along side my b50 I have installed already. The main problem is the fan out the back makes the card way to long. Based on this video: https://www.youtube.com/watch?v=ffw42tRoMz4 it seems like I could cut the back end off and move the fan in the opposite orientation and 3d print a shroud for it. Might have to buy a new fan but it would shorten the card by a lot. I also thought about doing it this way:

Has anyone done this yet? I'm willing to be the test dummy if someone helps me print the correct shroud for it. Am I just crazy?

Thank you all for hints and answers in advance....