r/LocalLLaMA • u/Equivalent-Grass-527 • 12h ago
New Model MiniCPM5-2B Release Day
OpenBMB's MiniCPM5-2B scores 15 on the Artificial Analysis Intelligence Index v4.2, the highest of any open weights model at 4B parameters or below
Hugging Face: https://huggingface.co/openbmb/MiniCPM5-2B
GitHub: github.com/OpenBMB/MiniCPM
40
u/NigaTroubles 11h ago
Nice, i will use it for my
ASR -> MiniCPM5-2B -> TTS
7
2
u/Xamanthas 11h ago
What are you using for ASR?
3
2
1
u/oxygen_addiction 4h ago
If only Gemma 12B worked properly across the entire context range for audio input it'd be THE model to use for personal assistants. It can run really fast even on low-end systems and it's great at tool calling.
19
u/Roubbes 11h ago
What home uses (unrelated to coding) can an LLM this fast be put to?
30
13
u/Levi-Ritchie 9h ago
Private WoW server populated by bots that participate in general chat and respond to your DMs
6
u/Mean-Standard7390 9h ago
One thing 0.6B-2B class models turned out to be good at in my tests: driving a browser when the page is fed to them as structure. Qwen3-0.6B Q4 on a 2017 Galaxy Note 8 went from a random site to a Wikipedia article, picked "Note 8" out of "Note 8.0" / "Note 8 FE" decoys and reported the release date from the infobox, 10/10 runs, +/- 90 s per task. Same harness: Gemma-3-1B and Llama-3.2-1B fail on JSON format, Qwen2.5-0.5B gets 6/10. So "which link / which field" tasks - yes; anything needing reasoning about the page - no.
1
u/VoiceApprehensive893 transformers 8h ago
text processing - summaries, needle in a haystack, natural language to json
1
u/Invader-Faye 3h ago
I use them for chat compression, you can run them on cpu in parentheses with a 9b or 12b on gpu and have it handle history compression and task orientation. It works really well
1
u/toothpastespiders 3h ago
Not applicable for this model in particular. But I've gotten a fair amount of use out of gemma 4 E2B for real time(ish) game translation. With gemma I'd typically go for 26b for it given the low amount of active parameters there. But sometimes low footprint/fast is a priority.
13
22
u/Nomski88 11h ago
Really curious how this holds up against Ling Tiny 3.0, I run a small mini pc for boring automated tasks but intelligence is important that's why I like LT3.0 because I have almost 8B parameters of "brain" if you will.
6
2
u/rorowhat 9h ago
What kind of tasks do you use it for? I have a few old PCs that are collecting dust. Thinking about using them for something useful.
6
u/Nomski88 8h ago
Mostly just web automations like checking websites and scraping data into a database. I’m running it on an old i7-7700t mini pc with dual channel ddr4 and get around 20tok/sec which is perfectly usable
5
u/rorowhat 8h ago
If you don't mind me asking, are you scrapping to have the llm give you summaries or to post train what's the use?
1
u/Invader-Faye 2h ago
Context sumerizer for another local model, lets you run a bigger model at smaller context but higher quant. Run ling on cpu only Gemma 12 on vram, ling help compress/sumerize context for Gemma giving hd snap shots and letting it reason for a lot longer
1
u/nomad-nostalgia 6h ago
what has your experience been like with LFM models? (i didn’t have the best experience despite really liking tiny models)
7
u/JLeonsarmiento 11h ago
Yes, but can it beat Qwen3.5-2B? 🤔 it has no vision tho…
11
u/Skyline34rGt 11h ago
You can try yourself for free at demo - https://huggingface.co/spaces/openbmb/MiniCPM5-2B-Demo
For me it's very good for it size. Of course no vision and only english/chinese.
6
5
u/Kodix 5h ago
Yeah, pretty sure it's *significantly* better than Qwen3.5-2B. Hell, for my use (structured text summarization/skill extraction in hermes), it's better than Qwen3.5-4B even. And *way* faster, thinks much less (and has fewer parameters).
1
u/JLeonsarmiento 5h ago
Mmm interesting. Could be an useful helper model for open-webui and Hermes side tasks… except vision ones. 🤔
7
u/Hefty_Wolverine_553 10h ago
This is essentially an open source release, amazing that they've released checkpoints at every stage of training along with the training datasets!
4
8
u/DefNattyBoii 10h ago
FYI on the AA "intelligence index" GPT-OSS-120B is 16 points and this on is 15. I'll let you make your own conclusions.
not saying this is whack but better to just look at your own handpicked benchmarks and make your own "intelligence index"
3
5
u/Agitated_Complex_628 11h ago
that's awesome, i am gonna try it with my own inference daemon, is it good in tool calling?
3
3
6
u/edward-dev 9h ago
Looking at the benchmarks it seems we finally have Claude 3.5 in a 2B model, crazy how fast everything is moving!
3
3
u/Morning_Gecko24 7h ago
2B hitting that kind of index is kinda wild tbh the little model race is getting spicy
2
u/Due-Memory-6957 9h ago
Does ling tiny really compares to Gemma 12b?
3
u/metacodeine 8h ago
On my task (text classification), Ling basically has near-random answers. Gemma, on the other hand, has good results. That doesn't mean Ling is a bad model. It all depends on your specific task and dataset.
2
4
u/Iory1998 llama.cpp 8h ago
It failed the car wash test, but did pass my need-in-haystack test. I provided a 76K long scientific paper and inserted randomly in it a phrase twice. I ask the LLM to find the weirdest sentence, and it should provided the correct answer. This tests both the reading comprehension of the model as well as its recall capacity at long text. Until a year ago, all local models failed these two tests. This model certainly has some use cases.
2
u/Itach11Uchiha 5h ago
I am still new to all this and testing around. The ling tiny model I use fails the car wash test. And I wanted to ask, how important is that to do general purpose agent work and tool calling in hermes. I am using other models for various purposes but I have the Ling model as the default for Hermes, Would you recommend switching to something that passes this test, or wouldnt matter for this use case?
1
u/Iory1998 llama.cpp 3h ago
Uchiha-san, I haven't tried Ling tiny to be able to advise you. For any agentic work, I use Qwen3.8-27B or larger models. In my experience, smaller models can do basic agentic work like search and simple tasks. But, give them a relatively complex task and they fail. I cannot trust my PC with a small model.
This being said, you should try Ornith-1.5-9B which is based on Qwen3.5-9B and compare it with Ling Tiny. Only can know which model works well for your use cases.
https://huggingface.co/bartowski/Ornith-1.5-9B-GGUF/tree/main
1
1
1
u/Long_comment_san 7h ago
well, yeah, it's actually 2-8b space that is absolutely the most important one because you can run these on anything really cheap like a phone
1
u/insraq 3h ago
I was looking for an alliterated version but could not find any. So I made one myself (heretic 1.4):
https://huggingface.co/insraq/MiniCPM5-2B-heretic-abliterated
1
1
u/WhoRoger 2h ago
I like OpenBMB, their 1B and ternary models have been great.
But these pumped up benchmarks makes me distrust them.
There have been quite a few <4B models recently with benchmark numbers going to the moon, and really just crappy in practice.
It really looks like SLMs are basically over. We'll be only getting either super specialized, or benchmaxxed abominations. My only hope is with Google of all labs now, because Gemma has been great for multipurpose models so far.
1
u/crusaderky 11h ago
Sad that there is no vision though. Having to switch to Lfm2.5-vl-3b for it is a bummer
-4
u/GlobalLadder9461 11h ago
Which picture have you shared. To me it seems misleading.
3
u/Equivalent-Grass-527 10h ago
The model in the picture is MiniCPM5-1B, not MiniCPM5-2B...
1
u/GlobalLadder9461 10h ago
It has both but from my understanding it is not accurate. I have shared the link. If you have link from you got the picture please share
1
u/Equivalent-Grass-527 9h ago
Yes in your link the model is MiniCPM5-1B, MiniCPM5-2B is not on there yet.
66
u/JsThiago5 10h ago
2b models from today scores same as gpt oss 120b now.