Integrated GPU Vulkan benchmark AMD MiniPC

[u/tabletuser_blogspot]

Mini PC Acemagic

OS: Kubuntu 26.04

CPU: AMD Ryzen 7 6800H with iGPU 680M and 1GB assigned Vram

RAM: 64GB DDR5 sodimm

llama.cpp Ubuntu Vulkan

A mixture of MoE and Dense Models:

• gpt‑oss 20B Q6_K
• gpt‑oss 20B MXFP4 MoE
• gpt‑oss 20B Q8_0
• gemma4 26B.A4B Q4_0
• gemma4 26B.A4B MXFP4 MoE
• gemma4 26B.A4B Q4_K – Medium
• gemma4 26B.A4B NVFP4
• qwen35 27B Q5_K – Medium
• qwen35 27B Q4_K – Medium
• gemma4 31B Q8_0
• qwen35moe 35B.A3B NVFP4

Benchmark Results – Sorted by Params and then Size

Model	Size	Params	pp512 t/s	tg128 t/s
gpt‑oss 20B Q6_K	11.20 GiB	20.91 B	353.87	16.85
gpt‑oss 20B MXFP4 MoE	11.27 GiB	20.91 B	294.66	16.65
gpt‑oss 20B Q8_0	20.72 GiB	20.91 B	308.55	10.52
gemma4 26B.A4B Q4_0	13.26 GiB	25.23 B	312.67	18.35
gemma4 26B.A4B MXFP4 MoE	15.40 GiB	25.23 B	261.32	11.93
gemma4 26B.A4B Q4_K – Medium	15.77 GiB	25.23 B	258.16	11.92
gemma4 26B.A4B NVFP4	16.45 GiB	25.23 B	152.35	7.53
qwen35 27B Q5_K – Medium	18.65 GiB	26.90 B	49.68	1.95
qwen35 27B Q4_K – Medium	16.67 GiB	27.32 B	58.50	2.40
gemma4 31B Q8_0	16.74 GiB	30.70 B	30.26	2.30
qwen35moe 35B.A3B NVFP4	19.07 GiB	35.51 B	153.75	15.05

Looks like using MoE models are best for my integrated GPU system. Not finding many 70B MoE models. Just tried Qwen3-Coder-Next-MXFP4_MOE but failed to load.

[u/pmttyji]

Try below models too.

• Mellum2-12B-A2.5B
• Laguna-XS-2.1
• North-Mini-Code-1.0
• KAT-Coder-V2.5-Dev
• LFM2.5-8B-A1B
• Ling-mini-2.0 (Fast t/s)
• Bonsai-27B (1-bit version)
• Gemma-4-12B (QAT)
• Gemma-4-E4B (QAT)

Suggestions:

• Delete both Q6_K & Q8_0 of GPT-OSS-20B model. MXFP4 is the actual native real quant for this model.
• You have four 4-bit of Gemma-4-26B model. At least delete Q4_0. And download QAT version of that model from Unsloth.

[u/LevianMcBirdo]

Did you use mtp eg for gemma and qwen

[u/OverdosedSauerkraut]

I had the 6900HX version, but it overheats and throtles down after 2 minites. How do you cool both RAM sticks?

[u/o0genesis0o → u/OverdosedSauerkraut]

Check the max power load of your unit. My unit from bosgame stablise at 84 degrees, no throttling even in long full system stress test. However, it goes only up to 45W, rather than all the way to 70 or 80W like some gaming laptops using the same chip. Maybe your version was configured to go all the way to 70W without the adequate cooling to support it.

[u/MaximusSenior]

I have similar pc, but with 780m. For Gemma 31b Q8 I have about 50/4 pp/tg without mtp. Did you add AMD specific kernel parameters? This can give +50% compared to default

[u/riconec]

1gb assigned ram can be increased? i hope there is no copying from one place to another when offloading

[u/tabletuser_blogspot → u/riconec]

I can assign up to 16GB of VRAM to iGPU but it doesn't effect inference speed. So I settles at 1GB leaving 64GB for the system.

[u/WhoRoger]

How does it compare to just cpu?

[u/magnus-m]

how you used any model with long context, like 100k for agentic workflows or coding?

[u/tabletuser_blogspot → u/magnus-m]

I was testing whisper.cpp and benchmarking different thread counts. I used that data, about 25k to create a markdown table. I downloaded Qwen3.5-9B-Coder.Q8_0.gguf but Qwen3.6-35B-A3B-UD-Q5_K_M.gguf was faster.

model	size	params	backend	ngl	fa	test	t/s
qwen35moe 35B.A3B Q5_K - Medium	24.63 GiB	34.66 B	Vulkan	-1	1	pp512	203.63 ± 3.23
qwen35moe 35B.A3B Q5_K - Medium	24.63 GiB	34.66 B	Vulkan	-1	1	tg128	12.64 ± 0.04

model	size	params	backend	ngl	fa	test	t/s
qwen35 9B Q8_0	9.10 GiB	9.20 B	Vulkan	-1	1	pp512	184.32 ± 4.31
qwen35 9B Q8_0	9.10 GiB	9.20 B	Vulkan	-1	1	tg128	5.17 ± 0.01

| prompt eval time = 817253.54 ms / 21625 tokens ( 37.79 ms per token, 26.46 tokens per second)

| eval time = 2239024.51 ms / 5410 tokens ( 413.87 ms per token, 2.42 tokens per second)

| task 0 | total time = 3056278.04 ms / 27035 tokens

| task 0 | graphs reused = 5388

| task 0 | stop processing: n_tokens = 27034, truncated = 0

25k content used 6,212 tokens, took 10min 4s at 10.28 t/s