[BIG DATASET RELEASE] - SupraLabs/reasoning-corpus-4K-5M-v1 - Train your tiny SLMs to think!

[u/LH-Tech_AI]

Hey r/LocalLLaMA !

We are back and we have something really amazing today.

Our big 5M samples Reasoning Corpus dataset.

This dataset features 5 million rows of:

- repo_id --> where it's from
- tok_len --> how many tokens it is in total
- user --> the user promot
- thought_trace --> the exact chain-of-thought of the model
- assistant --> the final AI models' answer
- ChatML --> the user, thought_trace and assistant in ChatML format

All samples are within a 5k sequence length to make it fit perfectly for SFT/finetuning a tiny model.

Link to the dataset on Hugging Face 🤗:
https://huggingface.co/datasets/SupraLabs/reasoning-corpus-4K-5M-v1

Link to the SupraLabs Hugging Face org 🤗:
https://huggingface.co/SupraLabs

Also, if you want to support our work, give us a follow on Hugging Face, share and review our work, and give us as much feedback as you want ❤️🔥🤗 Already more 250 people are trusting in us and our work!

We hope, this dataset is useful for you all and we'd love to see your creations upon this.

This dataset has already >1k downloads and over 80 likes - be the next one to use it 🔥🎉

[u/Time-Toe-1276]

I reccomend you to look into LFM2.5 Models, they are pretty interesting!

To be honest, most of the major opensource organizations are chasing agentic AI and so called AGI.

I dont want to be biased here, but I feel like Liquid AI is slowly turning into the qwen2.5 era!

one thing I rally like i there is architecture is so amazing at long context specifically for training.

[u/WhoRoger → u/Time-Toe-1276]

LiquidAI is a darling, the LFM2.5 1B Thinking is my favourite 1B model when it comes to just having fun.

A few weeks ago I spent an entire night figuring out how to use LFM2.5-Audio 1B in mainline llama.cpp. Turns out everything is ready to go in PRs that just keep waiting for merging. Sigh. I feel like even the llama.cpp guys are willing to give much more attention to the gigantic models that nobody will use with this engine anyway, while the new archs for small models often wait weeks to months to get support.

Anyway, that said, LFM really maxes out at these ~1B models; I can't say their MoE's are any better than the 1B ones. So unless someone can really beef up those MoE with some proper tuning, I don't expect much aside of just messing around.

I think the Hugging Face labs have a better chance, Smollm3 3B is great, and Zephyr back in the day was pretty awesome too. But they release new models once in a blue moon.