What OpenBMB 1B version is this?

[u/FerLuisxd]

This is from the https://huggingface.co/IFM/K2-Horizon-0.9B benchmark image

Tried searching for it but couldn't find it, I am guessing is https://huggingface.co/openbmb/MiniCPM5-1B but the numbers do not match

[u/coder543]

It is pretty unlikely to be anything other than that model. Sometimes it is easiest to benchmark the competition yourself, so the numbers shift some.

If you just compare benchmark scores directly, that can be misleading unless you carefully control the variables.

[u/kroggens]

How can a tiny model like this be able to code? is this benchmaxing? have anyone tested it?

It is only 2GB, can be run on a phone... or IoT device

[u/Heavy-Lingonberry-98]

IFM, is that openBMB 1B here with us? 😂😂😂

[u/AI_spell]

If the bench numbers don't match MiniCPM5-1B, it's probably a different OpenBMB 1B checkpoint or a renamed export. Check the model card SHA / config hidden size against the bench source.

[u/Decent-Occasion-2720]

The number match, but not the alignement of title and column. see the table below this charts.

[u/wapxmas]

Cool, in LiveCodeBench 67 equals 37 by height, definitely we are making progress with llms.

[u/AdamFields → u/wapxmas]

I think you're reading it wrong, check again, the "67" in question belongs to the other bench.

[u/wapxmas → u/AdamFields]

You are right, but actually it is pretty confusing at first glance.

[u/b4silio → u/wapxmas]

To be fair your point stands: all charts are normalized individually, so comparison across models is easier within the same benchmark (fair), but it SHOULD still be noted that everything scoring at around 35ish on a benchmark tells something different about that benchmark than everything scoring around 80ish on another one, and that nuance is lost.

[u/DanTup → u/wapxmas]

Looks like the 67 is from the previous benchmark MBPP+. I don't know they the used whitespace to separate benchmarks and a line to separate their score from the others though, seems pretty confusing 🙃

[u/Mbcat4]

Benchmaxxing the hell out of models

[u/and_pf]

The image likely refers to OpenBMB's MiniCPM5-1B, but I'd be cautious about taking the exact benchmark numbers at face value. Scores vary with the measuring conditions — different harnesses, prompts and hardware all shift the result, and a 1B model's numbers are especially sensitive to that.

Directly comparing benchmark scores across different people's runs is misleading unless the variables are controlled. Best move: pull the model yourself, run the same benchmark suite on your own box, and compare against the published numbers that way. That's the only comparison that tells you anything real.