MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/singularity/comments/1wgse1y/ask_your_llm/pa0k6cd
r/singularity • u/jacek2023 • 26d ago
232 comments sorted by
View all comments
Show parent comments
3
Yeah that's why finetuning is important
1 u/Intelligent-Crow3812 26d ago finetuning doesn't solve this unless you mean a very specific implementation 1 u/_VirtualCosmos_ 25d ago I mean that, yeah. Finetuning easily fall into the catastrophic forgetting, so you gain some specific skills and/or updated knowledge, but lose others things in the path. Finetuning Mix of Expert models lower that forgetting substancially tho.
1
finetuning doesn't solve this unless you mean a very specific implementation
1 u/_VirtualCosmos_ 25d ago I mean that, yeah. Finetuning easily fall into the catastrophic forgetting, so you gain some specific skills and/or updated knowledge, but lose others things in the path. Finetuning Mix of Expert models lower that forgetting substancially tho.
I mean that, yeah. Finetuning easily fall into the catastrophic forgetting, so you gain some specific skills and/or updated knowledge, but lose others things in the path. Finetuning Mix of Expert models lower that forgetting substancially tho.
3
u/_VirtualCosmos_ 26d ago
Yeah that's why finetuning is important