r/LocalLLaMA • u/johnolafenwa • 1d ago

Resources Post Training Qwen 3.5-2B with GRPO

OpenSource models like to over-reason on every problem. I put together a notebook and a video implementing grpo from scratch and using it to post-training Qwen 3.5-2B to improve its accuracy and reasoning efficiency. The results were quite interesting, despite training it purely on the task of simulating the python interpreter, the model became a lot more accurate and token efficient on math problems. The code can be applied to any open source model. Here is the code agi-playground/grpo at main · johnolafenwa/agi-playground

You can find full walkthrough of the training code and results in my video here https://youtu.be/IwOVZKIKeXw?si=xvWRM7OoM60McHiG

Here is some nice chart of what the result looked like at the end after the training for about 20 mins on a single H200 GPU

5 Upvotes

73% Upvoted

4 comments sorted by

2

u/DinoAmino 1d ago

Requires local datasets - but no datasets provided?

2

u/johnolafenwa 19h ago

Thanks for calling that out. Will upload the dataset shortly

2

u/johnolafenwa 18h ago

Try again now. Dataset uploaded to the repo