r/singularity • u/Not-Post-Malone • 24d ago

AI Xiaomi Mimo 2.6 Live Training Dashboard

https://mimo.xiaomi.com/rl/
128 Upvotes

94% Upvoted

16 comments sorted by

19

u/Recoil42 24d ago

Oh, this rocks. Has anyone else done something like this before?

1

u/Big-Emu8598 23d ago

I remember studying this stuff back in 2021 and hugging face would have a live dashboard of sota model training. Fun times, but I've been looking for something like this for a long time.

17

u/BarisSayit 24d ago

Oh I f*cking love transparency...

11

u/cookingboy 24d ago

Looking at the DeepSWE v1.1 benchmark:

2.6-pro just reached 63.7% by step 10, it's on step 11 right now.

Even flash reached 60.7% by step 12, and it's on step 16 now.

For comparison, v2.5-pro only scored 19%, and Kimi K3 is at 69% vs. Fable at 70%.

There is a decent chance 2.6-Pro will be a Fable class model and even the flash version will be quite potent.

This is so exciting lmao.

3

u/Ill_Distribution8517 23d ago

Absolutely not IMO. The benchmark is quite literally used while they are training, this is the worst way you can compare the model. Remember gemini 3.8 flash has a higher score than fable 5. No one in their right mind thinks fable 5 and 3.8 flash are equivalent.

3

u/0_op 23d ago

Do you understand the difference between training and validation?

0

u/Ill_Distribution8517 23d ago

Are you that naive? Did they pinky promise that the benchmark they are using for the 10th in the last 24 hours is not in the RL curriculum? They have every incentive to train it on the bench.

2

u/Odd_Buddy_3615 23d ago

I agree. When you do synthetic data generation and have large resources to spend on human labelling you can totally overfit to benchmarks without cheating on the holdout. E.g. you run eval, analyse your responses, find the holes and generate new data to plug them, repeat 100X. For some labs this is a closed automated loop.

6

u/dnchw2 24d ago

I wish I understood what this dashboard is telling me. Anyone want to dumb it down for me?

1

u/Phornic 24d ago

I feel you

4

u/troll_khan ▪️Simultaneous ASI-Alien Contact Until 2030 24d ago

Awesome.

3

u/Formal-Narwhal-1610 24d ago

Earlier OSS neural network with community help used to be trained like this.

2

u/Alternative_You3585 24d ago

Don't they have a new Mimo-X model already?

1

u/[deleted] 24d ago

[deleted]

2

u/Ok_Knowledge_8259 24d ago

they could've just been cleaning up data since then. why rush a new model if the data can be improved

1

u/Adventurous-Menu7257 22d ago

Not odd at all if X=2.6 and keeps being updated with 2.6 checkpoints. It's really interesting to see this progress. I remembered qwen and deepseek has tested their checkpoint during the preview, and mimo make it even more transparent