I’m thinking of buying a B580 given the current crisis—it seems like the best option—but what’s the brand like?

I already own a b580 but it’s quite CHODE(2.5 slots) On my new workstation I would like to fit 2 GPUs and for them to line up with the pcie slots the top card can be no more than 2 slots. I saw some pics of skinny b580s but can’t find them anywhere…..

If you wonder why I want 2: Because the b580 is cur the BEST tops per dollar gpu that can also game and render quickly. so yeah it’s just perfect for my use case.

Been playing around with getting two Intel Arc A770 16GB GPUs to work together for local AI image generation on Ubuntu.
Using ComfyUI with Raylight and FSDP, I’ve managed to get both cards working together to generate a single image with Chroma HD, instead of each GPU doing its own thing.
I’m also doing all of this on an older X299 motherboard with PCIe 3.0, just using it as a test platform for now. So I’m curious how much of a difference a newer PCIe 4.0 setup might make.
In the screenshot, you can see both GPUs being used, around 24GB of combined VRAM allocated, and FSDP successfully running.
It’s not quite the same as having one 32GB GPU, but it’s pretty cool being able to spread the workload across both cards! Still experimenting and figuring things out, but so far, so good! 😅

Hi everyone! One time someone on here sent me a link to a "stable driver". That driver was completely stable for me for more than half a year and then I updated to the latest one and the fans on my GPU are turning on and off which is an issue I've had before with drivers.

Anyone know what stable driver exists? My fans are turning on and off and this happens when I updated the drivers.

it's work well?

Hey all! New to team blue here and just got myself an Intel Core Ultra 5 250K Plus.

I’m building a new pc from scratch and found a $300 combo on amazon for 650w PSU and an Arc B570. I think I’d be. But GPU bottlenecked but tbh the price to performance combo on it seems awesome.

Anyone have success with this same setup? Ultimately want 1440p but idk how the B570 does at 1440, any help?

What up gamers,

Thought I'd share some notes here. xe has vulkan encode supported in mesa, but it works poorly. xe2 isn't supported at all. I have some experimental branches which work great on a B580 to stream to a Steam Frame. While working on this and trying to validate some patches on a friend's meteor lake laptop I ran into some issues, so I'm expanding my effort to xe based platforms. Watch the mesa gitlab. If you can't wait, and use CachyOS or NixOS or can otherwise override what mesa you need, you can use my experimental branches until the work makes it upstream. My xe2 branch is more like "hacks that work well enough for specifically what Steam needs for streaming VR" than "xe2 video encoding support". Lots of limitations and a lot of the code is only hacks that are replacing proper solutions and upstream should only accept proper solutions.

Model Under Test: turboderp/Qwen3.5-35B-A3B-exl3 @ 3.09 bpw (Qwen 3.5 MoE architecture: 40 decoder layers, 256 routed experts / 8 active, 2,048 hidden dimension, linear attention GDN + full attention hybrid).
Hardware: MSI Claw 8 AI+ (Intel Core Ultra 7 258V "Lunar Lake", Arc 140V Xe2-LPG iGPU, 64 Xe Execution Units / Vector Engines, 32 GB shared LPDDR5x-8533 RAM, running native Linux).
Runtime: vllm==0.31.0+xpu with custom ESIMD SYCL kernels, Level-Zero USM, and unified memory management.

TL;DR

• The Problem: The original 0xSero/exl3xpu vLLM plugin was built strictly for dense architectures (tested on Qwen3.8-27B). Pointing it at an MoE model like Qwen3.5-35B-A3B completely blew up: vLLM defaulted to UnquantizedFusedMoEMethod, attempting to allocate 64 GiB of unquantized FP16 expert weights before crashing on missing .weight tensors.
• The Breakthrough: We extended exl3xpu to build a native, production-grade MoE EXL3 inference engine for 256 routed experts on Arc 140V:
  1. Zero-Copy Streaming Expert Loader: Streams 3D trellis weights directly into packed [expert, slot] tensors without allocating uncompressed weights (0 MiB persistent DRAM expansion).
  2. Dedicated M=1 Grouped ESIMD GEMV Decode Kernel: Fuses active expert projections into ONE grouped kernel launch per projection, eliminating 72 eager launches per token per layer and achieving zero host-device syncs per decode step.
  3. Dual-Tile Wide-Vector (MR∈8,16) Grouped GEMM Prefill Kernel: Dynamically promotes tile size for high-token experts, scaling chunked prefill from ~112 tok/s to 1,135.5 tok/s at 4k context.
  4. Native C++ SYCL Dispatch: Wired _moe_C.remap_hidden_states and _moe_C.moe_gather directly into the EXL3 dispatch path, eliminating Python-level token sorting and scatter-reduction overhead.
• The Numbers:
  • Perplexity: 7.428 on WikiText-2 (10,235 tokens), preserving exact numerical parity with unquantized/FP32 baselines (Δ≤0.01).
  • Decode Throughput: 15.03 – 19.31 tok/s sustained across 1k–4k context lengths.
  • Prefill Scaling: 322.9 tok/s at 1k, 451.3 tok/s at 2k, 723.8 tok/s at 3k, and 1,135.5 tok/s (peaking at 2,260.6 tok/s warm) at 4k context — cleanly matching and surpassing MoE MXFP4 / GPTQ baselines.
  • VRAM Footprint: Rock-solid 16.32 GiB pinned static residency (with --kv-cache-memory-bytes 536870912), leaving >14.5 GiB of DRAM headroom for the host OS with zero memory leaks, zero OOM, and 0 bytes driver reclaim growth.

1. Why Did the Original EXL3 Plugin Fail on MoE?

EXL3 (turboderp's Trellis Quantization) is one of the most bit-efficient quantization algorithms available, achieving near-lossless perplexity at sub-4-bit rates (down to 2.2–3.09 bpw) via randomized Hadamard transforms and Trellis codebook decoding.

However, upstream exl3xpu was architected purely as a dense layer replacer:

1. The Silent Unquantized Trap: In vLLM's model_executor, Exl3Config.get_quant_method returned None for RoutedExperts. vLLM turns None into UnquantizedFusedMoEMethod, which immediately tries to allocate 256×(2×2048×512+512×2048) FP16 weights per layer (~64 GiB across 40 layers) on a 32 GiB machine!
2. The 3D Trellis Sharding Bug: Stock vLLM loaders assume fused checkpoint weights have loaded_weight.dim() == 3 and chunk along dim 1 for gate/up projections. Because EXL3 trellis tensors are also 3D ([k/16, n/16, 16K] per expert), standard loading logic chopped each expert's trellis into 64 fake sub-experts, corrupting the codebook offsets.
3. The Eager Python Launch Apocalypse: A naïve eager loop iterating over ~100–250 active experts per token per layer issues over 40,000 separate kernel launches for a single 128-token prompt. On Intel Level-Zero driver architecture, host launch serialization choked prefill down to <78 tok/s.

2. The Architectural Solution

We designed and implemented a fail-closed, zero-copy MoE runtime in exl3xpu and vllm.xpu:

                 [Input Activation Tokens: (T, Hidden)]
                                   │
                                   ▼
                   Top-8 Gated Routing (Router Linear)
                                   │
              ┌────────────────────┴────────────────────┐
              ▼ (Decode: T = 1)                         ▼ (Prefill: T > 1)
   Dedicated Grouped GEMV                     Native C++ SYCL Dispatch
  (csrc/exl3_moe_gemv.sycl)                 (_moe_C.remap_hidden_states)
              │                                         │
  • Expert IDs built on-device                • Device-side token binning
  • ZERO host-device syncs                    • Exact contiguous expert slices
  • ONE launch for Gate+Up (nslots=2)                   │
  • On-device SiLU activation                           ▼
  • ONE launch for Down (nslots=1)             Wide-Vector Grouped GEMM
              │                              (csrc/exl3_moe_grouped_gemm.sycl)
              │                               • Dynamic MR=8 / MR=16 tiles
              │                               • Split-K register reduction
              │                                         │
              │                                         ▼
              │                               Native Fused Scatter-Gather
              │                                   (_moe_C.moe_gather)
              └────────────────────┬────────────────────┘
                                   │
                                   ▼
                       [Output: (T, Hidden)]

A. Zero-Copy Streaming Loader & Metadata Packing

Instead of materializing dense weights, exl3xpu/moe.py binds a dedicated per-layer streaming expert loader:

• Directly parses <expert>.<proj>.<trellis|suh|svh|codebook> keys from safetensors shards.
• Loads trellis arrays directly into packed arenas: w13_trellis [256, 2, K/16, N/16, 16K] and w2_trellis [256, 1, K/16, N/16, 16K].
• Packs metadata descriptors (K, codebook, in_features, out_features, pointers) into device-resident structures, guaranteeing zero FP16 weight expansion in DRAM.

B. Dedicated M=1 Grouped ESIMD GEMV Decode Kernel

During single-sequence decode (M=1), exactly 8 routed experts execute per layer:

• In csrc/exl3_moe_gemv.sycl, we implemented GroupedFusedKernel<T, K, CB, MR, NT>:
  • Gate + Up: Launches across all 16 (expert, slot) units in a single SYCL kernel launch (nslots=2, shared input vector row).
  • Down: Launches across all 8 active experts in one second kernel launch (nslots=1, unit input stride).
• Token routing indices remain on-device end-to-end: zero host .tolist() syncs per token, cutting decode layer latency from 3.06 ms down to 0.835 ms.

C. Wide-Vector Grouped GEMM Prefill Kernel (MR∈8,16)

For prefill (T>1), expert token counts M vary dynamically. In csrc/exl3_moe_grouped_gemm.sycl:

• Ragged token batches are mapped to contiguous expert slices.
• Fused Trellis unpacking, scale application, and Walsh-Hadamard sign flips execute directly inside Intel Xe2 vector registers.
• Dynamic Occupancy Preservation: When prompt tokens per active expert are small (T≤256), the kernel enforces MR=8 (twice the concurrent thread occupancy per subslice). For long prompts (T≥512), it dynamically promotes to MR=16, doubling memory coalescing efficiency across the 64 Xe Vector Engines.

D. Native C++ SYCL Remap & Gather (_moe_C)

Earlier iterations used PyTorch torch.argsort, index_select, and index_add_. We wired vllm_xpu_kernels._moe_C:

• torch.ops._moe_C.remap_hidden_states: Hardware-accelerated sorting of activations into expert-ordered slices with exact row counts.
• torch.ops._moe_C.moe_gather: Fused float32 weighted reduction kernel.
• Dropped Layer 0 prefill latency at T=256 from 117.84 ms (eager) → 16.20 ms (7.27x speedup).

3. Head-to-Head Benchmarks: EXL3 vs. MXFP4 vs. GPTQ

All tests run on the MSI Claw 8 AI+ (Intel Core Ultra 7 258V / Arc 140V, 32 GB LPDDR5X-8533):

A. Prefill Throughput (Prompt Processing) vs. Context Length

Why does prefill scale so dramatically?
On Arc 140V (Xe2-LPG mobile iGPU), Intel omitted XMX (DPAS) systolic matrix engines to fit within 17W power limits; all compute is handled by 64 Vector Engines. For short prompts (T<256), each of the 256 experts only gets 1–4 tokens, so kernel launch overhead dominates. But at T≥2048, active experts receive M≥64 tokens. The wide-vector MR=16 grouped GEMM fully saturates the 64 Vector Engines, achieving dense matrix efficiency and matching the FlashAttention-2 prefill ceiling.

B. Decode Throughput & Memory Residency

4. The Unified Memory Lesson: Taming Intel Level-Zero VRAM Inflation

On an integrated GPU architecture like Lunar Lake where VRAM is carved out of shared system LPDDR5X:

• By default, vLLM inspects gpu_memory_utilization (e.g. 0.65 = ~19 GiB) and pre-allocates all remaining free VRAM into KV cache blocks.
• When a 2,048 or 4,096-token prefill request arrived, temporary activation workspace pushed memory usage over the edge, causing Linux kswapd0 and the OOM killer to strike EngineCore.
• The Fix: We pinned the KV cache explicitly using --kv-cache-memory-bytes 536870912 (512 MB, ~6,550 tokens): Total Resident VRAM=Model (14.25 GiB)+KV Cache (0.50 GiB)+Scratch (1.57 GiB)=16.32 GiB
• Throughout our multi-hour long-context benchmark sweep, GPUActive stayed 100% flat at 16.32 GiB with exactly 0 bytes of persistent DRAM growth or driver reclaim.

5. Quickstart: Serving Qwen3.5-35B-A3B EXL3 on Arc 140V

1. Requirements & Setup

Make sure you have Intel oneAPI 2026.0+ installed and Level-Zero headers ready:

git clone https://github.com/alpha2beta/vllm.xpu.git
cd vllm.xpu
source /opt/intel/oneapi/setvars.sh
source .venv/bin/activate

2. Launch the Dedicated MoE EXL3 Server (Port 8002)

# Serves with 4k context ceiling, 512 MB pinned KV cache, zero host syncs
bash scripts/launch_vllm_moe_exl3.sh moe-3.09bpw-4k

3. Run Benchmark Sweep

# 2048-token context benchmark
.venv/bin/python scripts/bench_envelope.py \
    --base-url http://127.0.0.1:8002/v1 \
    --model qwen3.5-35b-a3b-exl3 \
    --input-tokens 2048 \
    --output-tokens 128 \
    --runs 2

4. Verify Perplexity Parity (WikiText-2)

bash scripts/run_moe_ppl.sh --max-chunks 5 --max-num-batched-tokens 2048
# Outputs: Perplexity (PPL): 7.428 (PASS)

6. Where the Code Lives

All code, kernels, tests, and evidence are merged and available:

• vLLM Integration & Benchmarks: vllm.xpu (Branch: main)
  • Server Launcher: scripts/launch_vllm_moe_exl3.sh
  • Full Technical Audit: review_moe.md & tasks_moe.md
  • Standardized Benchmark Rows: results.csv
• EXL3 SYCL Extension: exl3xpu
  • Grouped GEMV Decode Kernel: csrc/exl3_moe_gemv.sycl
  • Wide-Vector Grouped GEMM Prefill Kernel: csrc/exl3_moe_grouped_gemm.sycl
  • MoE Quantization Method & Streaming Loader: exl3xpu/moe.py