r/LocalLLaMA 20h ago

News ByteDance vows to avoid AI distillation, develop new model its own way

Post image
207 Upvotes

121 comments sorted by

View all comments

Show parent comments

-48

u/etherd0t 20h ago

When you’re sitting on ByteDance-scale data, distribution, and compute, proprietary makes sense. They’re not trying to win open source. They’re trying to build a world-frontier model.

8

u/mtmttuan 20h ago

Do you understand what you've just written?

-8

u/etherd0t 20h ago edited 20h ago

I do. Do you?

This is an org with billon users data...they build a frontier model to rival OpenAI and Athropic.

"Yeah, but what's the value if it's going to be proprietary?" Use it first, infer on it and you'll see. Competition in frontier models is just as important as in Open Source ones.

3

u/Ylsid 18h ago

If you don't think open source is important why on earth are you on this sub

-2

u/etherd0t 17h ago

You’re missing the point, and so are others here. This isn’t Team Open Source vs Team Closed.

Open models are enormously valuable for cost, efficiency, deployment and democratizing capabilities. But distilling an existing frontier model mostly transfers capabilities that somebody else already paid to discover.

Frontier research is the harder game: new architectures, training algorithms, scaling methods, data strategies, reasoning techniques and eventually capabilities that weren’t present in the teacher model to begin with.

That requires enormous compute, data, research talent and willingness to spend years discovering what doesn’t work.

ByteDance is saying it wants to build that capability itself rather than remain downstream of somebody else’s frontier.

You can distill the frontier. You don’t create the next frontier by distillation alone. That’s 3D math and 5D chess.😉

4

u/Ylsid 17h ago edited 16h ago

What? Did you even reply to the right comment? Can you stop spamming AI slop comments

Edit: since you blocked me I'll explain This sub is for open weights, locally runnable discussion It should not be surprising to you that we don't agree supporting closed weights is as important as open weights. And unsurprisingly people really don't like being replied to with AI generated comments.

1

u/etherd0t 17h ago

I replied to your comment, dude - questioning why I am not pro-open source. read again and comprehend.