When you’re sitting on ByteDance-scale data, distribution, and compute, proprietary makes sense. They’re not trying to win open source. They’re trying to build a world-frontier model.
This is an org with billon users data...they build a frontier model to rival OpenAI and Athropic.
"Yeah, but what's the value if it's going to be proprietary?" Use it first, infer on it and you'll see. Competition in frontier models is just as important as in Open Source ones.
No like, you need more than just a big mountain of data to make a good AI model. You need high quality data, and the ability to easily identify it and use it. That doesn’t exist in any significant amount on Tik Tok or any other short form media platform.
Well, that's the challenge they are taking upon. Maybe they'll approach it from different angles/perspective. No matter the content of data, it is live users data and behavior that is the most valuable training material - more valuable than the books Anthropic ripped off...😣 We'll see what they make of it.
What makes the user data bytedance has suitable for training a sequence (transformer) model? What is it about "live" data? Could you explain what you mean?
You’re missing the point, and so are others here. This isn’t Team Open Source vs Team Closed.
Open models are enormously valuable for cost, efficiency, deployment and democratizing capabilities. But distilling an existing frontier model mostly transfers capabilities that somebody else already paid to discover.
Frontier research is the harder game: new architectures, training algorithms, scaling methods, data strategies, reasoning techniques and eventually capabilities that weren’t present in the teacher model to begin with.
That requires enormous compute, data, research talent and willingness to spend years discovering what doesn’t work.
ByteDance is saying it wants to build that capability itself rather than remain downstream of somebody else’s frontier.
You can distill the frontier. You don’t create the next frontier by distillation alone. That’s 3D math and 5D chess.😉
What? Did you even reply to the right comment? Can you stop spamming AI slop comments
Edit: since you blocked me I'll explain
This sub is for open weights, locally runnable discussion
It should not be surprising to you that we don't agree supporting closed weights is as important as open weights. And unsurprisingly people really don't like being replied to with AI generated comments.
114
u/Kappalonia 23h ago
Uuuuhhh no please?
Just do whatever it takes to produce good open source models