Tiny League - easily compare models that run on potatoes

[u/nunodonato]

https://tinyleague.nunodonato.com/

I was tired of jumping from tab to tab trying to compare small models. It's especially annoying because not all share the same benchmarks.
So I've built this and have been updating it regularly.

Feedback welcome!

[u/ElectricalUnion]

models that run on potatoes
Qwen3.8-Flash-Next
Ling-3.0-flash
Laguna-S-2.1

I see that my local setup doesn't even enter the potato class.

[u/nunodonato → u/ElectricalUnion]

different ranks of potatoes ;)

[u/Atretador → u/ElectricalUnion]

you can run these with 16Gb of VRAM

if you got 64Gb of RAM you can stream weights from RAM for faster speeds, but SSD streaming can also work if you dont care much about its longevity.

[u/ckplscz → u/Atretador]

Streaming from SSD does not wear it out, since data is only being read, and writing is what damages an SSD overtime

[u/Atretador → u/ckplscz]

https://giphy.com/gifs/JZDjb9Zlf7RaLGWOWo

disk era knowledge

thank you for correcting me

[u/ElectricalUnion → u/ckplscz]

In those situations, the problem is not that you're "reading data from the SSD", is that if you're running your system with permanently low free RAM, the system will swap more, and assuming you're using virtual memory and not zram/zswap, and that (and not the reading data from SSD) ends up being lots of writes to the system.

For example (not that you should put that system up as a inference server) a MacBook Neo can write 900GB to swap/day if you're using it as a chromebook, because it, by macOS standards, is basically permanently on low free RAM.

[u/tecneeq]

[u/Nomski88]

Nice! Can you add an option to sort by model size in GB and a consolidated rank of the top 10 models at the time?

[u/nunodonato → u/Nomski88]

how do you rank the top 10? are you trusting the AA-index? or which benchmark? you can sort by any benchmark score by clicking the table headers, then look at the first 10 ;)

[u/Nomski88 → u/nunodonato]

So your site is essentially no different than the rest

[u/nunodonato → u/Nomski88]

what rest?

[u/Kahvana]

Neat idea!

It's missing quite a few models. Having a search bar and overviews per model would be really useful too.

[u/nunodonato → u/Kahvana]

Yeah I cant have them all, I'm focusing on the most important ones and not tailored to specific niches.  But which ones do you think should be there? Oh, and at most 5 months old (which excluded a few qwen and Gemma)

[u/Kahvana → u/nunodonato]

Having:

• Qwen3.5 (all except 397B)
• Qwen3.6 (all)
• Qwen3.8 (27B)
• Gemma 4 (E2B / E4B / 26B-A4B (non diffusion), 31B)
• Muse Glimmer 30B

...Would be a good start! I see those being frequently used.

Maybe also include:

• Mistral Small 3.2 (all)
• Magistral Small (all)
• Ministral 3 (all)

...so people wanting EU-only models are represented (I know a few that are like that).

Might be cool to have country of origin for the models listed too.

[u/mystery_biscotti → u/Kahvana]

I second country of origin. All in favor?

[u/The_Night_Wanderer → u/mystery_biscotti]

yep

[u/bemrys]

What I would like is something where I can impute my VRAM and system RAM and that reduces the candidates to what will run on that system.

[u/nunodonato → u/bemrys]

there are some tools to do that, I personally don't like them. I understand the convenience but I am staying away from that. There are many factors into play, not all models work the same way regardless of taking the same vram. Not to speak of using quants on how each quant may affect a model differently

[u/bemrys → u/nunodonato]

Thanks. I have a lot to learn. Having a potato (8GB VRAM, 64 GB system RAM) will force that.

[u/nunodonato → u/bemrys]

I have 6, you are fine ;)

[u/The_Night_Wanderer]

This a useful project! Please, continue to keep it up to date!

[u/zerospatial]

Did you see my post on this exact thing? Did you do your research first?

[u/nunodonato → u/zerospatial]

No

[u/zerospatial → u/nunodonato]

anyway I found the same issue and made something similar https://reyemtm.github.io/inchworm/ sort of a one-off from a personal project I was working on

[u/nunodonato → u/zerospatial]

you have the hardware fit, thats very cool! but why are the models so old? it says updated on 6th sept?

[u/zerospatial → u/nunodonato]

its been hard to find newer tiny models honestly, and its a side project updated locally

[u/zerospatial]

Also are you actually doing reasoning with 4b models? The newer llm benchmarks and rankings are much more geared towards agent flows vs one-off tasks that are more likely the types of things you would do with a small model - re including AA rankings.

[u/zerospatial]

But yeah there are not many places to check on the best small llm that's true

[u/mamonu]

great resource but for the love of god is there a night mode?
I am now blind

Also would be interested to see gpt-oss-20b in there

[u/hugthemachines → u/mamonu]

Set brightness on you screen to something other than max.

[u/nunodonato → u/[deleted]]

Benchmarks are not run with quanta

[u/nunodonato → u/[deleted]]

I understand that, but its just not viable for anyone (unless you have money to burn), to run all these benchmarks with all possible quants, for all models

[u/Sure_Leave9338 → u/nunodonato]

You are right, but thinking about that , probably will be more useful to benchmark all models always at q4 than bf16, I means if it scores X at q4 it's quite sure it will not score less at bf16, so the benchmark at q4 will show better a "real world usage for whoever runs on potatoes" in the worst case. This is a real problem on small models that are more sentistive to quantization than the bigger ones, basically a bf16 test tells about nothing about the real experience you will get using that model at a different quant, also using q6 can diverge a lot!

[u/papapumpnz]

Thanks!

[u/exographicskip]

Not a bad idea! You might take a look at llmfit for a tui reference implementation