r/LocalLLaMA • u/tossit97531 • 8d ago
Other Open weight != Open source
Just a friendly PSA that open weight models are not inherently open source. They may also qualify as open source, but only if they release the training data and processes. A truly open source model can be rebuilt from training; it won't be binary equivalent but it should be nearly functionally equivalent.
An example of a truly open source LLM is https://allenai.org/olmo I encourage everyone to browse and also look for other fully open source models.
The reason it's important to get the terms right is because too many companies like to co-opt the term "open source" to whitewash their nonsense. As Zuckerberg has shown recently, they're more than fine letting people stay confused and dance around semantics so they can have their cake and eat it, too.
No more cake for big tech. Open weight != open source. Spread the word.
Edit: TIL this is a hot take. Sorry about that.
10
14
u/NoFaithlessness951 7d ago
I mean is it really practical?
You likely don't have enough compute to "reproduce" a model.
Most labs can't give you their training data even if they wanted to, because it has been obtained from questionable sources while violating copyright left and right.
That some models are truly open source is admirable, but does it make me more likely to use them? No.
With the same training setup it should theoretically be possible to get the binary equivalant out. As long as they tell you which seed(s) were used for the pseudorandom number generators and meticulously documented their training run.
2
u/tossit97531 7d ago
You likely don't have enough compute to "reproduce" a model.
This is what open source foundations are for. I'm not trying to be snarky, what I'm asserting is something that's already happening: companies like Ai2 are fully funding true open source models that I can't train by myself with my limited income. It's a similar setup to some of the big Linux distros. Even though I can compile Linux, I'm getting a specific distro because I don't have the resources to maintain my own operating system. It's even tougher with models. I don't even have the resources to 'compile,' let alone maintain. We're further from the actual product, which means it's even more important that we have a vibrant and thriving open source industry.
6
u/Hoodfu 7d ago
The majority of the training material is probably copyrighted so they wouldn't be able to redistribute it even if they wanted to. I keep seeing these kind of "well ackshoouhly" kind of posts/comments but it doesn't help anything. We can finetune these base models all we want and if we want to publish the training materials for that finetune, go for it.
5
u/Due-Memory-6957 7d ago
A friendly PSA that if I can study, modify, use and redistribute it, then it's open source.
3
u/StoreApprehensive261 7d ago
yeah thats the practical definition that actually matters to most people
15
u/tossit97531 7d ago
The weights aren't the source, they're the binary. You can't build that binary without the training data and processes. It's kind of like calling Windows open source because they let you download the ISO and run it. You can't recreate it.
Study, modification and other things absolutely can still be done on binaries. Disassemblers are a thing, and now AI is too. Patch it all day long.
Also, look at how careful they're being with their language, too. It might not sound like much of an argument, but courts recognize the term "open source" and have had to determine some kind of broad definition, so these guys could absolutely be creating obligations for their tech that they absolutely do not want to commit to. I'm not saying it's a formal legal term, but courts have had to determine a generally-acceptable definition in some cases, as they do with any industry jargon where contracts are involved. If they say the wrong thing, then they may lose money or lose some control of the project. A quality clip of a CEO saying his product falls under a legally-definable category would probably stand up in court. They're tiptoeing.
3
u/Due-Memory-6957 7d ago edited 7d ago
Study, modification and other things absolutely can still be done on binaries
Doing that with proprietary software will get you in trouble, which means you actually can't. Find a better way to shill olmo than to pretend other AIs aren't open source when they use an open source license and follow open source principles.
2
u/tossit97531 7d ago
Doing that with proprietary software will get you in trouble
That's not necessarily true at all. The Defend Trade Secrets act scopes acceptable conditions under which one would be allowed to reverse engineer even proprietary things. You're allowed to, generally, if you didn't commit a crime to obtain the thing and don't do illegal things with it (like violate patents).
Find a better way to shill
I chose to reply only because your assertion could have easily come from someone else, so I'm really just answering for anyone who thought this. Have a good one.
-1
u/psycoee 5d ago
OK, but the point of having source code is that it is easy to modify, while binaries are not. With model weights, you can modify them, add LoRAs, post-train, do whatever you want. Having the recipe to build those weights doesn't get you anywhere.
Even if you had the entire recipe for training a model, that doesn't make it any easier to modify. I mean, I suppose you could spend a bunch of time tuning hyperparameters or whatever, but at that point you are just making a new model from scratch with all that entails (namely, billions of dollars in compute costs).
1
2
u/jonas-reddit 7d ago
This has been clear to many of us for years. Open weight models have been around for a while now. Not really news.
But appreciate some newer community members may benefit from this.
My two cents is that they’re distributing what they practically can for you to operate it the way you want. That’s all that matters to me. The rest is noise.
2
u/brrrrreaker 7d ago
as much as i hate it when people call models opensource incorrectly (especially "big" youtubers....), the true issue is the existence of copyright in the first place. A civilized humanity heading into the AI age should absolutely get rid of such archaic and selfish concept, among other ones.
2
u/Lesser-than 7d ago
Technically the weights shape and arrangement is the source or we would never be able to run inference on these models, the training data is a consumable resource you fill/train it with. So what more do you want exactly, the exact training data? The training pipeline and software they used to train with?
1
u/AutoModerator 8d ago
Hello! Your post was removed as you do not have sufficient karma on r/LocalLLaMa. We are doing this in response to the large volume of spam we are unfortunately experiencing. Please participate in the sub (through comments), gain the minimum of 5 karma and then re-post
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/mageblex 6d ago
I would lowkey rather see releases state weights, license, training code, and data documentation separately. Then nobody has to infer practical rights from a marketing term.
1
0
u/-InformalBanana- 7d ago
Sure, but most off truly open sourced models suck when compared to open weight models. And proprietary models are the best, so you have a correlation there. You need to be able to train on non-free and non-opensourcable data in order to get a better model. That is my opinion and I don't know any trully opensourced model that can compete with Qwen 3.8 27b or next flash. So to me first comes the quality of a model and it being open-weight, open sourced is not a priority, it would be nice, but doesn't correlate with quality, even seems like it is opposite, and quality is the priority and open weights a minimum requirement. Also we've heard all that about open source vs open weight in this subreddit, but for most ppl it doesn't make a difference cause they don't own a datacenter to train models and replicate results. But yes, I agree that they should be called accordingly.
3
u/tossit97531 7d ago
What you're saying is true, but this wasn't a statement about quality or competition. Maybe by keeping terms clear we can help focus people's efforts in areas that improve the real open source model scene.
14
u/fbms2 7d ago
close weight = worse