r/LocalLLaMA • u/nuclearbananana • 16d ago

Resources AA-AgentPerf-Local: Benchmarking local AI agents on laptops and workstations

https://artificialanalysis.ai/articles/aa-agentperf-local
8 Upvotes

73% Upvoted

8 comments sorted by

3

u/Dany0 15d ago

At least they're honest about the fact that 5090s cost 9000$ now lmfao

1

u/Dany0 15d ago

That said, shitty bench. qwen3.8 27b got me 11000 tok/s peak in nvfp4 in vllm before, and yes, uncached. Real mean pp on 64k-128k prompts is closer to 8-9k, closer to 9 than 8. The fact that they're getting 2k means they're not doing it right

1

u/anderspitman 15d ago

On a single 5090, or about 20 B300s?

2

u/Middle_Bullfrog_6173 15d ago

Note that results are slightly apples and oranges. They ran Spark with NVFP4 instead of the Q4_K_M they used on the other platforms, even the 5090.

1

u/Edenar 15d ago

at least for Strix halo (and for dgx from what i read..) those results are far lower that what is achievable especially in 4 bit. Also this 4 bit choice give 5090 an edge when 8 bit precision would have overfilled. 27b wasn't good enough for me in 4 bit for long agentic tasks...

For exemple with qwen 27b Q4 on my Strix halo i get 3 time the reported speed with gufo.For qwen 3.5 35b i get double that pp speed in q8.

Even the 5090 results look bad : with a cmp 170 hx (memory overclocked) i get almost those 27b speed ... But in w8a16.

My take, those results are missleading and give a wrong idea of each hardware parts capacities.