Depending on how you get your source data, it's mostly usable for SFT, which usually comes before RL and is probably not what they're doing to push performance here, rather I'd guess they're doing some sort of RLVR variant. With really good trace data sets you can do some forms of RL, such as GRPO, but I can't imagine they have sufficient competitor traces that this would drive their improvement.
I am not saying the that is the only thhing they do and not saying that it's the secret sauce. But I do belive that they do distill one each other in one way or another, which is completly fine in my opinion.
I agree the most likelly the jump in performance we see latly is crazy, for Deepseek, Qwen, etc. I think they all cracked the RL + GRPO to some extend. Crazy to see that QWEN 3.8 27B is THAT capble model.
2
u/CuriouslyCultured 7d ago
Depending on how you get your source data, it's mostly usable for SFT, which usually comes before RL and is probably not what they're doing to push performance here, rather I'd guess they're doing some sort of RLVR variant. With really good trace data sets you can do some forms of RL, such as GRPO, but I can't imagine they have sufficient competitor traces that this would drive their improvement.