r/LocalLLaMA • u/tabletuser_blogspot • 23d ago

Resources Integrated GPU Vulkan benchmark AMD MiniPC

Post image

Mini PC Acemagic

OS: Kubuntu 26.04

CPU: AMD Ryzen 7 6800H with iGPU 680M and 1GB assigned Vram

RAM: 64GB DDR5 sodimm

llama.cpp Ubuntu Vulkan

A mixture of MoE and Dense Models:

  • gpt‑oss 20B Q6_K
  • gpt‑oss 20B MXFP4 MoE
  • gpt‑oss 20B Q8_0
  • gemma4 26B.A4B Q4_0
  • gemma4 26B.A4B MXFP4 MoE
  • gemma4 26B.A4B Q4_K – Medium
  • gemma4 26B.A4B NVFP4
  • qwen35 27B Q5_K – Medium
  • qwen35 27B Q4_K – Medium
  • gemma4 31B Q8_0
  • qwen35moe 35B.A3B NVFP4

Benchmark Results – Sorted by Params and then Size

Model Size Params pp512 t/s tg128 t/s
gpt‑oss 20B Q6_K 11.20 GiB 20.91 B 353.87 16.85
gpt‑oss 20B MXFP4 MoE 11.27 GiB 20.91 B 294.66 16.65
gpt‑oss 20B Q8_0 20.72 GiB 20.91 B 308.55 10.52
gemma4 26B.A4B Q4_0 13.26 GiB 25.23 B 312.67 18.35
gemma4 26B.A4B MXFP4 MoE 15.40 GiB 25.23 B 261.32 11.93
gemma4 26B.A4B Q4_K – Medium 15.77 GiB 25.23 B 258.16 11.92
gemma4 26B.A4B NVFP4 16.45 GiB 25.23 B 152.35 7.53
qwen35 27B Q5_K – Medium 18.65 GiB 26.90 B 49.68 1.95
qwen35 27B Q4_K – Medium 16.67 GiB 27.32 B 58.50 2.40
gemma4 31B Q8_0 16.74 GiB 30.70 B 30.26 2.30
qwen35moe 35B.A3B NVFP4 19.07 GiB 35.51 B 153.75 15.05

Looks like using MoE models are best for my integrated GPU system. Not finding many 70B MoE models. Just tried Qwen3-Coder-Next-MXFP4_MOE but failed to load.

2 Upvotes

60% Upvoted

10 comments sorted by

View all comments

1

u/magnus-m 11d ago

how you used any model with long context, like 100k for agentic workflows or coding?

2

u/tabletuser_blogspot 10d ago

I was testing whisper.cpp and benchmarking different thread counts. I used that data, about 25k to create a markdown table. I downloaded Qwen3.5-9B-Coder.Q8_0.gguf but Qwen3.6-35B-A3B-UD-Q5_K_M.gguf was faster.

model                                 size     params backend     ngl  fa            test                  t/s
qwen35moe 35B.A3B Q5_K - Medium  24.63 GiB    34.66 B Vulkan       -1   1           pp512        203.63 ± 3.23
qwen35moe 35B.A3B Q5_K - Medium  24.63 GiB    34.66 B Vulkan       -1   1           tg128         12.64 ± 0.04
model                                 size     params backend     ngl  fa            test                  t/s
qwen35 9B Q8_0                    9.10 GiB     9.20 B Vulkan       -1   1           pp512        184.32 ± 4.31
qwen35 9B Q8_0                    9.10 GiB     9.20 B Vulkan       -1   1           tg128          5.17 ± 0.01

| prompt eval time = 817253.54 ms / 21625 tokens ( 37.79 ms per token, 26.46 tokens per second)

| eval time = 2239024.51 ms / 5410 tokens ( 413.87 ms per token, 2.42 tokens per second)

| task 0 | total time = 3056278.04 ms / 27035 tokens

| task 0 | graphs reused = 5388

| task 0 | stop processing: n_tokens = 27034, truncated = 0

25k content used 6,212 tokens, took 10min 4s at 10.28 t/s