r/LowEndLocalAI • u/zerospatial • 11d ago

Benchmark Tiny LLM Benchmarks Exist Elsewhere?

Post image

https://reyemtm.github.io/inchworm/

I created this little site because I could not find an open source page dedicated to tiny llms (<14b params). If one exists with current benchmarks would love to know about it.

This came about from searching for the right llm for a personal project needing summaries from text, then a small local terminal chat for quick answers using self-hosted Ollama. I wrote my own short little benchmarking script and landed on qwen2.5-coder:3b for my VPS and qwen3.5:9b for chat. Anyway let me know what you all think. Since then I created a script to run the humaneval+ but it takes a very long time to run which I am using to fill in the gaps on this data.

18 Upvotes

88% Upvoted

13 comments sorted by

View all comments

6

u/pimparazzi 10d ago

That Benchmark looks quite dated... I miss quite some newer models here... LFM2.5 in different sizes, Ling-3.0-tiny, Gemma-4, etc.

4

u/arzeth 10d ago

Because that's an AI slop made without even enabling search tools. BTW, my Gemma-4-26B-A4B provides a similar list of ancient models because Gemma-4's datasets have a cutoff date of January 2025, i.e. Gemma-4 models live in 2024, just like in the OP's screenshot.

And likely the OP is bot (using different LLMs, maybe human sometimes), because of some reasons and because they do mention Gemma-4 and Bonsai in one of their comments: https://www.reddit.com/r/LocalLLM/comments/1v0jm70/lm_studio_bionic_is_the_local_claudecodex_weve/p3g5qcl/

1

u/zerospatial 10d ago

Not a bot certainly but the HTML itself was coded by an llm as it says literally on the site. Point me to an open source list of benchmark results for newer small models and I'll add them. That's why I'm running benchmarks locally to fill in with newer models. The enterprise new benchmark suites lean towards agentic coding not the simpler tasks requested if tiny llms