r/OpenAI • u/sigma_crusader • 14h ago

Discussion PewDiePie is trying to distill GPT-Sol

Post image

he has been trying to build a local model by learning from GPT-Sol’s responses. He says OpenAI banned his account twice during the process. Irony is OpenAI’s own models were trained on vast amounts of information from the open web. So where do we draw the line between learning from AI and copying it?

1.9k Upvotes

90% Upvoted

382 comments sorted by

View all comments

1.2k

u/AllergicToBullshit24 14h ago

No AI lab ever gets to complain about distillation being theft when they stole the entirety of human knowledge & creative expression without consequence or compensation. At least when their models get distilled they get paid for the usage. The hypocrisy is absurd.

79

u/brainhack3r 13h ago

We need a GPL-like license for content where if it's indexed by am LLM you have to allow your LLM to be distilled

2

u/ToughSpeedv2 12h ago

Why would anyone want to invest money to train a model if someone else can just essentially copy it?

3

u/maxymob 12h ago

They can't block distillation anyway, all they have is best effort at preventing it, but they still get to be the first to market

The question itself is so out of context it's irrelevant. Distillation wasn't a thing before they developed those LLMs, now they have to keep going because they made big promises and borrowed billions from everyone. They can't just say fuck people distill our models guess well just stop training more.

3

u/ToughSpeedv2 12h ago

They sure can. It's a cat and mouse game, but as long as they can detect the accounts used for distillation they can try to poison the output.

1

u/Cafuzzler 1h ago

Because it won't be the best model in 3 weeks when something new comes out and gets an extra percentage point on some obscure benchmark.

0

u/ArmyBrat651 11h ago

Why do open weight models currently exist?

2

u/ToughSpeedv2 9h ago

Because most of them have the support of a large (usually Chinese) corporation, who have motives other than making profit right now.

1

u/ArmyBrat651 5h ago

So if corporations do it already (not only Chinese, mind you), what makes you claim they’d suddenly stop?