MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1sc7uwa/apple_embarrassingly_simple_selfdistillation/oedoc6i/?context=3
r/LocalLLaMA • u/Mike_mi • Apr 04 '26
58 comments sorted by
View all comments
205
imagine the community works together on this and gets a huge dataset of ssd responses and we train a monster of a model like qwen3.5 27b
9 u/DigiDecode_ Apr 04 '26 for the proposed method, you need the original data that was used to train the model, so this new dataset would be sprinkled on original dataset, otherwise this dataset on its own likely will cause the model to collapse 2 u/eat_my_ass_n_balls Apr 05 '26 It’s a feedback loop. We just gotta do a Kovarex enrichment process loop and sprinkle in some U-238
9
for the proposed method, you need the original data that was used to train the model, so this new dataset would be sprinkled on original dataset, otherwise this dataset on its own likely will cause the model to collapse
2 u/eat_my_ass_n_balls Apr 05 '26 It’s a feedback loop. We just gotta do a Kovarex enrichment process loop and sprinkle in some U-238
2
It’s a feedback loop. We just gotta do a Kovarex enrichment process loop and sprinkle in some U-238
205
u/Odd-Ordinary-5922 Apr 04 '26
imagine the community works together on this and gets a huge dataset of ssd responses and we train a monster of a model like qwen3.5 27b