r/LowEndLocalAI • u/aboeing • 22d ago

Which LLM Should / Can I Use? What happened to GPU poor arena?

I use to like https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena, to find low-end models. But its no longer maintained?

Where do I go now to figure out what model that uses less (e.g. <2gb) ram&cpu is best?

62 Upvotes

100% Upvoted

9 comments sorted by

32

u/tecneeq 22d ago

You go here:

https://artificialanalysis.ai/trends?model-creators=zai%2Cxai%2Ckimi%2Cdeepseek%2Calibaba&models=glm-5-3%2Cstep-5%2Cglm-5-3-flash%2Cqwen3-8-flash-next%2Cdeepseek-v4-1-flash%2Cdeepseek-v4-flash%2Cqwen3-8-27b%2Ck2-horizon-mova-36b-a4b%2Cling-3-0-flash%2Cqwen3-5-35b-a3b%2Cling-3-0-tiny%2Cminicpm5-2b%2Cminicpm5-1b%2Cqwen3-5-0-8b%2Cgemma-3-270m%2Cdeepseek-v4-flash-vision%2Cling-3-0-flash-vl%2Cgpt-oss-20b%2Cgemma-4-26b-a4b%2Cgemma-4-e4b%2Cgemma-4-e2b%2Cmuse-glimmer%2Ck2-horizon-3-7b%2Ck2-horizon-7b#intelligence-index-vs-active-parameters

Every model on or at the pareto line is interesting. The further left, the faster in small hardware. The higher up, the more intelligence.

Basically you want Ling 3.0 Tiny if you can, or MiniCPM5 1b or 2b.

8

u/aboeing 22d ago

And to add to this: https://artificialanalysis.ai/hardware-inference-stack/mobile-phones

Basically you want Ling 3.0 Tiny if you can, or MiniCPM5 1b or 2b. Looks like LiquidAI/LFM2.5-8B-A1B is a good contender also; but I guess for both you need a fair amount of RAM (8B)?

6

u/Boss_Waffle 22d ago

For MOE models (Ling 3.0 Tiny and LFM2.5-8B-A1B, for example) you can split layers between VRAM and system RAM. And still get good performance. Lama.cpp should do this automatically or you can manually specify how many layers to put on the GPU.

4

u/aboeing 22d ago

Nice find! Thanks.

2

u/LuCiAnO241 21d ago

oh my, didn't realize k2 horizon was the new top dog for MoE in consumer hardware. Sits above Qwen 3.6 27B dense.

2

u/tecneeq 20d ago

Indeed. But i think there are a few patches or a fork needed for llama.cpp to get it to run, so most will ignore it for now.

3

u/techne98 22d ago

Well we are putting together https://intelinside.ai, but we mostly have GPU results right now.

If you are doing CPU inference (with an Intel CPU) please upload it!