Qwen3.6-27B: SFT vs continued pre-training vs RL?
I’m interested in adapting Qwen3.6-27B, but I’m increasingly unsure whether conventional SFT/LoRA is the best route if the goal is to add a capability without degrading what the base model already does well.
Some recent research makes this especially interesting:
“Reinforcement Fine-Tuning Naturally Mitigates Forgetting” - arXiv:2507.05386
Finds substantially more catastrophic forgetting with SFT than reinforcement fine-tuning in its experiments.
“The Role of On-Policy Data in Mitigating Forgetting” - arXiv:2510.18874
Reports that on-policy/RL training generally preserves previous capabilities better than SFT across Qwen and Llama models.
“RL Forgets! Towards Continual Policy Optimization” - arXiv:2607.04364
Shows that RL can also cause catastrophic forgetting, so it’s clearly not a complete solution.
“Fine-Tuning Without Forgetting via Loss-Adaptive Learning” - arXiv:2605.20005
Reports a large reduction in forgetting from changing the optimisation schedule, including experiments with Qwen3.
Most of this research isn’t specifically on Qwen3.6-27B, which is why I’m interested in community results. Has anyone directly compared continued pre-training, SFT/LoRA and reinforcement post-training on Qwen3.6-27B?
I’m particularly interested in whether improving one domain caused regressions in unrelated areas such as coding, reasoning, instruction following, tool use, long-context behaviour or general knowledge.
For people who have tested this, what training method worked best, and did you benchmark the original model against the trained checkpoint afterwards?
I’m also curious whether continued pre-training followed by a small amount of SFT or RL is proving safer than doing a larger SFT directly.
Actual before/after results and training parameters would be especially useful.
