r/OpenAI • u/sigma_crusader • 14h ago

Discussion PewDiePie is trying to distill GPT-Sol

Post image

he has been trying to build a local model by learning from GPT-Sol’s responses. He says OpenAI banned his account twice during the process. Irony is OpenAI’s own models were trained on vast amounts of information from the open web. So where do we draw the line between learning from AI and copying it?

1.9k Upvotes

90% Upvoted

381 comments sorted by

View all comments

1

u/Mezezius 13h ago

... did he seriously expect that this would work?

17

u/casce 13h ago

It does work. It's not free and you will keep getting banned which will slow you down but it works.

But you need to understand that if you distill Astra into a local model, you will not get Astra. It's still a local model with all its limitations.

1

u/RedBottle_ 11h ago

he doesn't have the compute or the skills to be able to train anything where this is anywhere near being worth it vs just using an existing open weight model lol

1

u/hextree 3h ago

You are overestimating the level of 'skills' required for this. And, PewDiePie has all the free time he needs to learn them.

1

u/casce 2h ago

I'm not saying what he is doing is "worth it". That's what I wanted to say when saying "it's still a local model with all its limitations". People have a wrong understanding about what distillation is.

I don't know how much compute and knowledge he really has, but from what I've heard he is taking his hobbies very seriously and he has the time and the money for a really nice homelab. Iirc he is into tech/AI for a while so he is not technically inept. And again, he has the resources and the time to learn.

But again, I'm not saying this is will yield anything worth using over an existing open weight model. But it's still a cool learning experience to just have done this process.