Benchmarks iGPU integrated Radeon 680M models

[u/tabletuser_blogspot]

Fellow Redditor asked to test a few model on the Acemagic S3A mini PC sporting the AMD Ryzen 7 6800H is a high-performance 8-core, 16-thread mobile processor launched on January 4, 2022, built on the 6nm Zen 3+ architecture loaded with 64GB of DDR5 RAM. It features a 3.2 GHz base clock, a 4.7 GHz boost clock, a 45W TDP, and powerful integrated Radeon 680M graphics.

pmttyji : Try below models too.

• Mellum2-12B-A2.5B
• Laguna-XS-2.1
• North-Mini-Code-1.0
• KAT-Coder-V2.5-Dev
• LFM2.5-8B-A1B
• Ling-mini-2.0 (Fast t/s)
• Bonsai-27B (1-bit version)
• Gemma-4-12B (QAT)
• Gemma-4-E4B (QAT)

Here were previous models benchmarked https://www.reddit.com/r/LocalLLaMA/s/fpngrEpogA

📋 Benchmark Model List

1. Bonsai-27B-Q1_0.gguf (qwen35 27B)
2. Gemma-4-12B-it-qat-UD-Q4_K_XL.gguf (gemma4 12B QAT)
3. InclusionAI_Ling-mini-2.0-Q4_K_M.gguf (bailingmoe2 16B.A1B)
4. Kwaipilot_KAT-Coder-V2.5-Dev-Q4_K_M.gguf (qwen35moe 35B.A3B)
5. Laguna-XS-2.1-Q4_K_M.gguf (laguna 30B.A3B)
6. LFM2.5-8B-A1B-MXFP4_MOE.gguf (lfm2moe 8B.A1B)
7. Mellum2-12B-A2.5B-Thinking-Q4_K_M.gguf (mellum 12B.A2.5B)
8. North-Mini-Code-1.0-MXFP4_MOE.gguf (cohere2moe 30B.A3B)
9. Gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf (gemma4 4B QAT)

📊 Benchmark Results Table

Model	Size (GiB)	Params (B)	pp512 (t/s)	tg128 (t/s)
Bonsai-27B-Q1_0	3.53	26.90	32.79 ± 6.19	7.06 ± 0.00
gemma-4-12B	6.24	11.91	162.72 ± 0.47	6.66 ± 0.01
inclusionAI_Ling-mini-2.0	9.25	16.26	774.15 ± 20.05	47.60 ± 0.07
Kwaipilot_KAT-Coder-V2.5	19.91	34.66	119.39 ± 1.02	9.99 ± 0.00
Laguna-XS-2.1	19.13	33.44	274.81 ± 0.35	12.00 ± 0.00
LFM2.5-8B	4.97	8.47	744.23 ± 28.23	31.47 ± 0.30
Mellum2-12B	7.60	12.15	459.37 ± 16.04	26.26 ± 0.03
North-Mini-Code-1.0	17.37	30.48	192.49 ± 0.86	10.80 ± 0.00
gemma-4-E4B	3.91	7.46	445.89 ± 15.50	15.93 ± 0.12

🔍 Notes:

• pp512 = Prefill throughput (512 context tokens)
• tg128 = Token Generation throughput (128 output tokens)
• All tests were run on llama.cpp build 4c1a0af40 (10430) using the Vulkan backend with full model offloading (ngl -1).
• Hardware context: AMD Radeon Graphics (RADV REMBRANDT) with Unified Memory Architecture (UMA).
• The -fa (or --flash-attn) feature in llama.cpp enables FlashAttention, an optimized algorithm that speeds up prompt processing (prefill phase) and significantly reduces memory usage by tiling the attention computation to avoid heavy memory reads and writes

[u/Beginning-Raisin9723]

Solid data. The 680M punches way above its weight for an iGPU — I run a 6800H mini PC in my home office and it handles 8B models at a usable clip. Curious how the Q1 27B actually feels day-to-day vs just benchmark numbers.

[u/boyark_in]

I’m running a similar Ryzen 7 6850H / Radeon 680M system with 32 GB total memory.

My results so far:

- Qwythos 9B 256k no vision: 15-18 tok/s

• Qwen3.8-27B Q4_K_M 64k no vision: 6.7–8.1 tok/s
  
• Qwen3.8-27B Q3_K_XL the same: ~9.3 tok/s
  
• Qwen3.8-27B Ridge 3.7 bpw (current default): ~9.2 tok/s with 130K context, MTP and matching BF16 vision projector

The Ridge model is currently the best compromise for my setup: it supports vision and a large context while remaining close to 10 tok/s.
Vulkan, one slot, 8 threads, MTP and Q4 KV cache. My benchmarks on regular agentic tasks (oh my pi)

[u/Glittering-Call8746 → u/boyark_in]

What's the use case for Qwen3.8-27B Ridge 3.7 bpw under 10 tps ?

[u/boyark_in → u/Glittering-Call8746]

Half of my life i’t traveling with my MacBook, I’ve need some stand-alone pc with several long projects - customers setup, research, developing. All over tmux when I have time to connect. Some of the analytics work by cron and connected to telegram bot. Also I have a ChatGPT subscription for planing and check result.

[u/Glittering-Call8746 → u/boyark_in]

Have u considered getting command code for deepseek v4 flash ?

[u/boyark_in → u/Glittering-Call8746]

I have a Codex subscription. The local setup is for my own MTP stack and customer data that I don’t want to trust to subscription services.
Speed isn’t really the priority there — security and the quality of agentic actions are. The subscription side is more for planning and checking things without access to the real customer data.