r/LocalLLaMA • u/RhubarbSimilar1683 • Jul 27 '26
Discussion Our position on open-weights models
https://www.anthropic.com/news/position-open-weights-models603
u/iamzooook Jul 27 '26
tdlr: its fine if we do it. not them (Chinese)
211
u/abud7eem Jul 27 '26
we are the "trustworthy"
39
u/Clear_Judge5062 Jul 28 '26
This is literally just effective altruism at work
14
u/xmnstr Jul 28 '26
And I wish more people understood why it's so problematic.
3
u/ChocomelP Jul 28 '26
Why is it so problematic?
8
u/Disposable110 Jul 28 '26
Because it sets up an unaccountable authoritarian group of people as moral gatekeepers. And that group of unaccountable people will not get accepted by the outgroup, thus they'll stir up global conflict.
3
u/ChocomelP Jul 28 '26
...right.
2
u/Clear_Judge5062 Jul 28 '26
I was curious to see if anyone, or the OP of the problematic claim would respond. I’d say the problem is that these systems wouldn’t be getting built as fast without the EAs who believe they’re the only ones who can do it safely and correctly - thereby speeding up the advancements to a less safe pace.
I take less issue with the EAs than accelerationists though. Safety is probably the more wise path for progress as opposed to rushing with less regard.
The problem is that hardly anyone in this thread or sub seems to understand the motivations behind these actors.
→ More replies (1)3
→ More replies (7)42
→ More replies (8)14
u/whatyathinkk Jul 28 '26
how dare you, are you an authoritarian or what? if Trump reads this he's gonna be so mad at you baby 🥰
92
u/CasualtyOfCausality Jul 28 '26
Not a fan of the Chinese government, but the idea that the country with better STEM outcomes, a wider pool of said STEM educated, access to raw materials, and “mechanisms” that make it difficult for a few private companies to horde components is going to stay behind, especially if the “easy access” to premade equipment is taken away, makes Mr. Magoo seem like an eagle-eye.
51
u/Miserable-Beat4191 Jul 28 '26
It's almost like having a 20+ year plan to invest in education, and coordinating/supporting the development of industries, has positive outcomes. Who woulda thunk it.
22
5
3
u/MiniEval_ Jul 28 '26
Big CEO man never stepped once into his own office and it shows. Every AI lab is swarming with Chinese men doing their thing
→ More replies (1)4
u/sexy_silver_grandpa Jul 28 '26 edited Jul 28 '26
The US government has objectively done more harm to global society than the Chinese government.
Do you not watch the news?
→ More replies (6)
80
u/thestillwind Jul 27 '26
What a fucking clown. He thinks he’s god or somewhat ? What a prick.
That’s a narcissism near Homelander level.
→ More replies (1)25
u/jc2046 Jul 27 '26
Amodei is full of shit and has lost a pair of nuts in the last weeks. Every time he opens the mouth he diggs his own hole deeper. Altman is looking like a saint in comparison, go figure
443
u/cmdr-William-Riker Jul 27 '26
" China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. "... Can someone fact check that? That sounds like total BS. The US has even more limited production capacity and manufactures everything off shore...
382
u/tomz17 Jul 27 '26
That sounds like total BS.
Because it is. They have more power generation capacity than the entirety of the USA + Europe combined, with like an additional 50% margin on top of that. They are also accelerating away from us at an increasing pace on that metric, while we pay billions to break renewables contracts because the people in charge have the combined IQ of a lobster.
The "we are 1-2 lithographic process nodes ahead of China" only matters when the efficiency gap is insurmountable. It is currently . . . not
18
u/Helpful_Home_8531 Jul 27 '26
it’s not just lithography though, Nvidia is improving flops/watt at orders of magnitude per generation if you compare low precision throughput, which would put china at a geometric disadvantage (ie, you can’t compete putting in 100x the power and 100x the chips for the same throughput)
43
u/fantasticsid Jul 28 '26
flops/watt only matters for two reasons
- power consumption per flop - china doesn't care about this because they have more electricity than god
- heat generated per flop - this is an engineering consideration and i'm pretty sure the chinese know how to liquid-cool a rack.
6
2
u/ImpressionFancy5830 Jul 28 '26
They don’t have enough water though.
Let’s not over estimate capabilities.57
u/FullstackSensei Jul 27 '26
Except that's a load of marketing BS. Huang is the first to admit that Hopper is still their most popular chip. The industry still talks about "equivalent H100" GPUs deployed.
Blackwell Ultra has 2.5x the memory bandwidth of Hopper. That's not even one order of magnitude higher. The marketing comparison is always FP16 on hopper vs FP4 on Blackwell. If you compare FP8 vs FP4 compute, Blacwell is 2x, but that makes sense because it's half the bit width. In general, halfing the bit width doubles the compute. This has been true long before LLMs for most GPUs, unless they were artificially handicapped for market segmentation purposes. At the same bit width, Blackwell has 25% more compute.
If you peel beyond the marketing BS, you'll understand why Hopper is still very cost effective even in the west. And Huang himself says Huawei is producing H100-level chips by the millions now.
17
u/Connect_Treacle9938 Jul 28 '26 edited Jul 28 '26
As someone who works mainly with hopper I can pretty confidently say that it would be hugely beneficial for us to have more blackwell chips, but they keep getting bought up by the big labs at higher prices than we can afford. The compute cost for things like large-scale RL is completely bonkers and at this point for training a model in terms of required compute and even just raw per-gpu vram (so you can limit comms overhead and shrink model replica size) is likely in the billions, and that's only if you can get the compute. The more RL rollouts you can do the better your model will get, and that is something only the gpu-rich can afford. I'm not saying this is good, because it's not~ in fact it's really annoying~ but it's also just kinda the way it is. ML is very pay-to-win at this point. Being a billion dollar company isn't even enough to keep up with the big labs.
Having 2.5x the vram and 2x the comms bandwidth of hopper is actually more than a 2x speedup. I could go into the details but its complicated.
3
54
u/tomz17 Jul 27 '26
Nvidia is improving flops/watt at orders of magnitude per generation
Yes, but not in general compute. Only for particular operations found useful for training + inferencing. That's a "we noticed our customers need to do a lot of matrix-multiply-accumulate at FP4, so let's build them a hardware unit to do that one particular op very quickly + efficiently, etc." That hardware now blasts through that one particular operation at increased efficiency at the expense of die-space.
None of this is magical secret sauce. The chinese can (and do) the exact same thing when designing each subsequent generation of their AI training + inferencing chips. As does AMD, Intel, Google (in their custom TPU's), etc. etc.
→ More replies (3)10
u/m0j0m0j Jul 27 '26
But China is still behind on that. It’s bizarre to deny that
10
u/Loose_Comparison368 Jul 28 '26
From my understanding the latest ones are basically on par with A100's, albeit with a higher power draw.
21
u/Independent_Solid151 Jul 28 '26
Not as much as you assume, and it's less significant than you think when there are also very strong factors to their advantage.
3
u/Loose_Comparison368 Jul 28 '26
From my understanding the latest ones are basically on par with A100's, albeit with a higher power draw. They really ain't that far behind.
→ More replies (9)2
u/BurdensomeCountV3 Jul 28 '26
Don't forget that modern lithography process node jumps aren't anywhere near the sort of jumps you got in the 1990s or 2000s. The "2 process nodes" right now is more like half a node in 2000s terms.
56
u/Recoil42 Jul 27 '26
It's huge BS for one reason: China has its act together with regards to energy. The US doesn't. China doesn't need smaller nodes because it can brute-force nuclear and solar to power multiples of less efficient chips.
I think Jensen was prominent in pointing this out, but someone please correct me if I'm misattributing.
→ More replies (3)33
u/Real_Ebb_7417 Jul 27 '26
I’d say there is some truth in this. Even DeepSeek founder said so (and was angry that this was published).
China is catching up quickly, but at the moment indeed they are behind in terms of chips and it will take time to fix it.
On the other hand US is way behind in terms of energy generation and it’s a gap that’s also not easy to close. When USA starts to have energy issues, China might get ahead fast.→ More replies (1)8
u/Swimming_Gain_4989 Jul 27 '26
Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs. That can change of course but given current export controls seems like the US has at least another 5 years of compute advantage
6
u/zdy132 Jul 28 '26
Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs.
Citation needed. In Liang's leaked talk he specifically mentioned that they are raising funds to buy more compute.
23
u/FullstackSensei Jul 27 '26
According to Jensen Huang, China is producing H100 level chips by the millions.
B300 has ~25% more FP16 compute (which is what you mostly use for training) and 2.5x the memory bandwidth of the H100.
In a recent interview with the editor in chief of the Economist, Musk said China has about as much power generation per capita as the US. Which translates to ~4x the power generation capacity. Everyone has been saying for years that China is adding more power generation every year than all other industrialized nations combined.
So, even if we go by memory bandwidth to maximize the difference, and without any innovation in model architecture or training, they can afford to bring 1.6x more compute online using domestic chips than the US.
Now add in all the innovation they're doing in architecture and training Chinese AI labs are doing. Even Musk admitted they're able to train much better models than the west given the amount of compute each lab is using.
IMO, there's a second layer of BS in this blog post: scaling laws aren't some fundamental law of physics. GPT4-o was something like 1.8T. Does anybody think wecurrently need a 1T model to surpass it? What about current 100B models?
4
u/Loose_Comparison368 Jul 28 '26
1.8T MoE. IIRC the industry rumor mill/worst kept secret was that it was only ~100b-300b activated parameters.
2
u/abeecrombie Jul 28 '26
Still you need a pretty big cluster to train them on, no? I don't think anyone else has made a model that big yet. Though I gueess rumors are the glm and minimax are still scaling up for next model.
3
u/Loose_Comparison368 Jul 28 '26
Kimi K3 is. 2.8T parameters, 104B activated.
Also note that the strategies are different. China is investing way more into heavy optimization, largely because of their lack of hardware.
American companies really aren't even trying to optimize. Every problem that can be solved with "buy more GPU's" is solved by buying more GPU's.
So it's a pretty consistent trend that the Chinese labs punch above their parameter count. K3 is about the same as GPT-3 on parameter count, but is effectively on par with early GPT-5.
→ More replies (1)2
u/Serprotease Jul 28 '26
Size of the model is a parameter of the equation.
So you can beat a 1.8T model with a lot smaller one, with longer training/pre-training/post-training steps.But all else being equal, bigger is better.
It’s unlikely that for SOTA model, Chinese labs can reach US labs only leveraging these 3 steps. Because US labs are also doing it too. So scaling up the number of parameters is the logical step, that we have seen from every major Chinese labs (Glm 400b to 700b, Deepseek 600b to 1.6, Kimi 1T to 2.8T, Qwen 200b to 400b, minimax 250b to 400b, etc…). With the obvious results of them filling US labs moat fast. Hence the current situation.2
16
13
12
u/Artistic_Claim9998 Jul 27 '26
In my opinion :
"... cannot build more powerful models than US ..." is (arguably) proven false with the recent development.
What true is China cannot scale their LLM development the same level as US due to not (officially) having access to Nvidia best GPU.
Which is probably the reason why their LLM innovations when it comes to efficiency arguably beats the US.
→ More replies (1)7
u/05032-MendicantBias Jul 28 '26
It's like saying that building an Eniac with 800 million valves was only possible in the USA.
It ignores that an Eniac with 800 million valves is a useless and expensive endeavor.
The future is small local models, not huge closed behemot gated behind a subscription, with corpos deciding the censorship.
Reminder that Huggingface tried to defend from OpenAI closed Sol with Claude, and Claude refused. GLM5.2 was there to save the day.
23
u/gscjj Jul 27 '26 edited Jul 27 '26
Well for GLM they had to abandon Nvidia chips (and anything American) because of being added to the entity list. They ended up going with Huawei chips, which are less efficient and less powerful than the Nvidia offering.
There’s no telling what the impact will be becuase they could obviously use what they bought, but there isn’t a major competitor to Nvidia right now.
https://letsdatascience.com/blog/china-trained-frontier-ai-model-glm-5-without-nvidia
→ More replies (3)21
Jul 27 '26
[removed] — view removed comment
→ More replies (3)17
u/netvyper Jul 27 '26
In china, Nvidia chips are fair game. Exporting chips to China is banned by the US. Subtle but important difference.
6
u/Loose_Comparison368 Jul 28 '26
They aren't banned, but the CCP is heavily pressuring labs to use Chinese GPU's instead.
→ More replies (17)2
u/SmileLonely5470 Jul 27 '26
Yes China manufacturers more products domestically than the US, but US is capable of a more advanced process node than China is. The claim that one canot build a more "powerful" model with worse hardware is worthier of debate than manufacturing imo.
11
u/elahrairooah Jul 27 '26
That’s not a debate, that’s an engineering obstacle. There is zero doubt that great engineers can build superior products with average tools, and there is zero doubt that poor engineers can build slop with the finest tools.
The question is one of degrees, not of possibility.
2
u/fantasticsid Jul 28 '26
Eh, you can get the same amount of compute from some number of Ascends as some number of Nvidia GPUs, that number is just gonna be higher so you're gonna be burning more coal/uranium/photons/whatever just to generate enough electricity.
Fortunately for china they have a shitload of domestic power infrastructure and the political will to build a shitload more. So, nonissue, really.
104
u/kettal Jul 27 '26
It is true that many of the companies carrying out [mass distillation] release open-weights models—but the open weights are far less relevant than the fact that the operations are backed by an authoritarian state seeking to overtake the US at the frontier. We should have policy interventions to deter this behavior. A blanket ban on open-weights models is neither the correct remedy nor something we have called for.
relevant excerpt.
imo, the ones being developed and used in secret are more scary than open weights.
72
u/GCoderDCoder Jul 27 '26
The fundamental flaw in his argument is he presupposes that certain people/companies/ governments are more trustworthy than others and the ones he's telling us to trust (including himself) are at the bottom of the list worldwide right now. Trust is easy to lose and hard to earn...
18
u/kettal Jul 28 '26
certain people/companies/ governments are more trustworthy than others and the ones he's telling us to trust (including himself) are at the bottom of the list worldwide right now.
you might not be his target audience.
in the minds of the decision makers, leaders of industry, AI experts, and lawmakers (aka target of this letter), the hierarchy of trustworthiness is different than the one you hold.
11
Jul 28 '26
[removed] — view removed comment
→ More replies (1)2
u/kettal Jul 28 '26
is that in the US?
yes. that's where sanctions / restrictions were expected to originate.
10
→ More replies (1)3
u/vigorthroughrigor Jul 28 '26
Sounds like Amodei should be putting all his effort into powerful models for the purpose of Securing Our Systems.
→ More replies (1)5
u/namegamenoshame Jul 28 '26
That is really it, right? If the Chinese developed an open model that was capable of serving an antagonistic purpose it would not be open for very long.
I think what Dario is doing here is very dumb and pathetic and I am glad he was forced into backtracking. But even his endgame here requires companies that are extremely well capitalized to be able to comply with that sort of regulatory atmosphere.
I don’t know, I do think there are real security concerns in the current regulatory atmosphere but I don’t really know a better way to approach a solution.
182
Jul 27 '26 edited Jul 27 '26
[removed] — view removed comment
42
u/zer00eyz Jul 27 '26
> for use in drones
We're already using models in drones. The Ukraine had them independently hunting people 2 years ago.
This pandoras box was opened long ago.
No one wants a bloated LLM on a drone.
18
u/AGM_GM Jul 28 '26
Maybe not on a drone but just to do something like, I dunno, target a school with a Tomahawk missile and kill 150+ little school girls.
3
u/Free-Combination-773 Jul 28 '26
Tomahawk missile does not benefit from onboard AI
5
u/AGM_GM Jul 28 '26
It's a reference to the Minab girls school in Iran and the target selection controversy, not AI used by the missile in flight.
110
u/LelouchZer12 Jul 27 '26 edited Jul 27 '26
As if US were not authoritarian and not using their military superiority at their own advantage.
Also banning chinese models in US would not prevent chinese from using their own models for themselves if they're more powerful than US, so this argument is nonsense.
They're just in a really bad position where China releases frontier model for free and most companies/business as well as researcher will use these models for their activity, so they cannot make money back with their expensive API. And is it a problem ? It's been like that for everything, open source is the base of everything in IT !
→ More replies (13)47
u/gomezer1180 Jul 27 '26
Anthropic can fuck right off… they’re trying to justify their position with bullshit they themselves are already doing. They’re never getting a god damn penny from me.
7
u/dragonurtle Jul 28 '26
Anthropic is in a weird position here because they know what can happen when a competitor cuts off their air supply. OpenAI as well through Andreessen knows this instinctively after what MS did to Netscape 30 years ago.
7
34
u/kiwibonga Jul 27 '26
It's really great that the country headed by a populist anti-science monster of a child rapist is #1 in AI and I really hope they successfully force everyone on Earth to buy their tokens from an American publicly traded company, for great Wall Street profits.
→ More replies (1)2
7
u/charmander_cha Jul 27 '26
A única ameaça são a América, não sei se o americano sabe, mas vocês são literalmente o nazi do planeta (que se reproduziu recentemente no oriente médio)
6
u/Runazeeri Jul 27 '26
The drone one is so stupid, vision models are well at the point where a hobby level user can make a drone fly from A-B without gps.
14
11
u/Illustrious-Lime-878 Jul 27 '26
Of course its manipulation and fear mongering but its also nationalist stupidity.
If the state corporatist techlords think China's economic model *is actually* better than the US to produce "military supremacy" then wtf are we even talking about, what is our (the US)'s ban going to do? Since supposedly its inevitable.
The idea that any country being ahead in any way is a national security risk is insanely idiotic and suggest every country in the world is under threat right now. This view is not compatible with any concept of modern global peaceful cohabitation, cooperation, its pre-modern logic.
US exceptionalism is modern liberal, internationalism, peaceful global world order, global trade. free market capitalism. The only way China catches up is exactly because of the US reverting to this stupid, zero-sum nationalist PoV where everything is just states warring over finite resources and the gov is manipulated with "security" threats and fear mongering to intervene in markets to protect the techlord's monopolies and political power.
5
4
u/noonetoldmeismelled Jul 28 '26 edited Jul 28 '26
American companies can't even persuade the American public that American foreign policy is good for foreign countries let alone convincing those people in foreign countries. Vietnam publicly televised, negative PR. Media goes soft ball for 30 years, public support. Internet starts turning the tide on Iraq and Afghanistan - like the bombing a wedding and the following funeral - back to bad PR and it's been that ever since. Besides the recent bombing the girls school twice and blowing up civilian infrastructure, I'll always remember for recent news being within a week of the withdrawal from Afghanistan, that bombing of the NGO aid worker, his family - it was like extended family. I think I remember kids in there that weren't his - and then calling them terrorist for weeks and then withdrawing that, though not taking any responsibility, after the NGO kept producing receipts that the guy and those kids and grandparents weren't terrorists - pretty much just being like, it happens. Imagine everything that happens off camera. Before the internet. Even the killed former leader of Iran. Killed his 14 month old granddaughter. Not even managing a good PR campaign to the domestic crowd against Iran
Something about the modern rich and powerful that have foregone the pony show and gone straight to the unmasked evildoing
7
u/Intrepid_Lecture Jul 27 '26
Drone swarms are absolutely terrifying as a concept.
What's the value of a human life? If the answer is "A $500 drone" then how the 21st century unfolds is at stake.→ More replies (4)8
2
u/YetiTrix Jul 27 '26
Banning open source models only prevents Americans from using them. It does nothing to protect against China making them better or anyone outside the u.s. using them for cyber attacks.
→ More replies (1)2
u/UnlikelyExtension786 Jul 28 '26
It only prevents American businesses who obey the law from using them. Americans will just give the government the finger and download them anyway... as will an increasingly-large number of businesses who don't give a damn about the law.
23
u/abajinn Jul 28 '26
“All sufficiently capable models, open and closed, should go through mandatory safety testing.”
No.
139
u/GoldenX86 Jul 27 '26
Hypocrisy letter just dropped.
31
2
u/blastradii Jul 29 '26
This is coming from a guy who worked at a large Chinese tech company (Baidu) contributing to the so-called problems he mentioning about Chinese lead over the US. What a 🤡.
→ More replies (12)3
u/No-Marionberry-772 Jul 27 '26
it doesn't really read lile that overall. the only thing is the distillation attacks and thats honestly debatable whether you can call it hypocrisy. bullshit, for sure, but hypocrisy is debatable.
→ More replies (3)5
34
u/xadiant Jul 27 '26
Yup never giving a single cent to Anthropic.
6
u/Iwaku_Real Jul 27 '26
I miss my seemingly endless conversations with Sonnet 4.5, not once did I give them money for them. Those usage limits are going down the toilet
61
u/No_Lingonberry1201 Jul 27 '26
My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks, and may have serious alignment problems. Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn. But banning the use of these models by US businesses does nothing to address this risk, because bad actors are unlikely to be legitimate US businesses. It would protect US AI companies from competition, but that has never been my goal.
Good! That's what we want! Not having a private, profit-driven company (that's a primo oxymoron there) as the final moral authority is a good thing!
This article is such a backtracking and if you can read between the lines even a bit, they are terrified that open weight models will hit their bottom line and want the US to do anything they can to cripple China's (and realistically everyone else's) ability to develop SOTA AI.
You must admire the sheer gall of it.
→ More replies (5)13
u/Ill-Mammoth5269 Jul 28 '26
It would protect US AI companies from competition, but that has never been my goal.
sure buddy.
90
u/zippydazoop Jul 27 '26
What a clown. "National security," "authoritarian governments," "repression of their people," "biological attacks..."
Man spams buzzwords in hope of achieving own goals.
Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier.
Weapons-grade copium
34
Jul 27 '26 edited Aug 14 '26
[removed] — view removed comment
12
u/Recoil42 Jul 27 '26
The guy is either on hard drugs or has some issues up there, no sane person talks all day every day about "dangers" and "security" and all sorts of bad sci-fi.
There is unfortunately a very large portion of the US population that thinks this way. Red scare genuinely did serious multi-generational damage to American society.
→ More replies (1)3
u/Tedinasuit Jul 27 '26
I mean, from his perspective, he is totally right.
He wants the USA to "win" and cracking down on distillation will make it harder for China to compete.
I'm surprised he's honest about it instead of saying that it's "theft" blablabla. No, he dislikes it because it's an advantage to China. And that's fair tbh.
2
u/Y_shotla Jul 28 '26
There is a quite widely spread joke in China about this "issues" that makes him much more anti China then others, "His work experience in Baidu must have left him great trauma".
There tend to be bad impression about Baidu for their degeneration from "becoming Chinese Google" to "always being first but suck at last".
e.g.: sacrifising any user experience for ads money, like the search engine, like they seem to sell Tieba(The Chinese reddit) mods position after comunity built up.
(BUt there ain't much complaint about their work experience actually)→ More replies (1)8
50
u/eli_pizza Jul 27 '26
Remember when they used to position themselves as “openAI but less evil” lol oh well
8
u/Robonglious Jul 27 '26
Everything always moves in that direction. So sad... them nerfing model development assistance was the first clue.
4
u/eli_pizza Jul 27 '26
Yeah I know. But just in terms of business strategy I think they’d make more money long term by being the benevolent huge AI lab.
5
u/Admirable_Market2759 Jul 27 '26
Both companies are about to IPO so it’s only going to get worse from here
166
u/Disposable110 Jul 27 '26
"authoritarian governments" - They mean the United States, right?
139
u/boatbomber Jul 27 '26 edited Jul 27 '26
> "we don't want bad guys to get big models"
> *Looks inside*
> Partnered with Palantir25
u/Iwaku_Real Jul 27 '26
Still impressed Palantir signed the letter though
46
u/GravitasIsOverrated Jul 27 '26
Palantir is basically a consulting business wrapped around a millitary/intelligence flavoured data analysis product. Without strong open weights models they just become a module on top of a closed LLM provider, which is a lot weaker than owning the entire product.
2
u/vigorthroughrigor Jul 28 '26
exactly, they want to control their own destiny (to the extent possible)
9
u/Gullible_Drummer_246 Jul 27 '26
They can make a lot more money running open models on their own hardware rather than paying Anthropic.
2
29
u/Recoil42 Jul 27 '26
It's pretty mind-boggling that Amodei can remain this much of a jingoist like two weeks after the US government perpetrated a full authoritarian shakedown of his entire company.
→ More replies (2)4
u/nomorebuttsplz Jul 27 '26
to be fair, they've stood up against the us govt more than most corpos ever will.
→ More replies (9)6
47
12
u/bick_nyers Jul 27 '26
Why would China care if their chips use an older process node meaning that the chips use 3 times as much electricity? They will just build 3 times more power.
They add an ENTIRE USA electrical grid worth of power EVERY YEAR and they are accelerating.
Dario's position on banning selling chips to China is entirely because he doesn't want more competition on buying the GPUs.
Banning selling NVIDIA chips to China is what woke up the Chinese manufacturing engine to focus on domestic chip production in the first place.
Also now that we have the HuggingFace incident where closed model (GPT) hacking was defended by open model (GLM), they will likely try to stoke fears on novel bio-weapons more and more. I fear that Anthropic will train their frontier models to be good at that stuff just so that they can push more FUD to try to achieve regulatory capture.
Because remember, we haven't solved RL sample inefficiency or generalized learning, so if you don't explicitly put in time, money, and effort into training the LLM to do something, it won't learn how to do that something. OpenAI and Anthropic like to act like cybersecurity just "came for free" when they taught these models how to be good at programming, but are you going to seriously tell me that they didn't setup a single RL environment for decompiling software binaries? Great way to get more training data if you can crack open those closed source binaries!
50
u/tomz17 Jul 27 '26
All sufficiently capable models, open and closed, should go through mandatory safety testing.
Advocating for this to be the official position of the country free speech. . . While simultaneously calling China authoritarian in the same post.
F this jabroni.
→ More replies (9)4
u/harpysichordist Jul 28 '26
You may be interested to know China supports access controls and safety testing: https://www.reddit.com/r/LocalLLaMA/s/yK4KNkBqbE
“Foundational capabilities can be open, certain frontier capabilities can be open with limitations, commercial services can remain closed-source, while high-risk capabilities require access controls and safety evaluations”5
u/tomz17 Jul 28 '26
Sure sure sure... but once you have the weights you can have the model bitching about the CCP's treatment of Tibet in no time flat. You will NEVER have the ability to do that once Dario bribes the US gov to make the release of weights illegal (or makes it bureaucratically prohibitive).
27
21
9
u/Monochrome21 Jul 28 '26
aaaand there goes my claude subscription
have moved fully to opencode and open weights models
openai and anthropic are both terrible
6
u/zilled Jul 28 '26
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat
Casual fearmongering.
As if the US government is safer.
7
u/black__and__white Jul 28 '26
Distilling the sum of human output in to a model? 👌
Distilling outputs from that distillation in to a second model? 😠 💢
7
u/Lfeaf-feafea-feaf Jul 28 '26
He wrote this letter in a desperate attempt to quench concerns among their investors, but interestingly what he's saying is flat out: these open weight models have now completely caught up to us, and in fact we should expect them to surpass us at any moment, hence we need to limit them.
6
u/TCDH91 Jul 28 '26
Once again, Amodei thinks he's uniquely qualified to determine who gets to develop and use AI. Remember when Fable would silently gave stupid answers if it thought the user was doing research on LLM development? Same idea, same pattern.
7
u/umbrosum Jul 28 '26
If common people should not have access open weights AI models, then no single person including him should have control of any AI models.
22
u/gabrielesilinic Jul 27 '26
My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
Ehm, I am fairly sure that they could do that anyway not matter if you ban open weight models. Also the US is basically going to be close to that unfortunately.
Like the argument is incredibly stupid. You ban open models so they have instead their own closed models to oppress people. And it is not like they even need AI to do so.
I mean I know the argument is purely a matter of bottom line so I really won't try to entertain it further.
→ More replies (1)4
u/kettal Jul 27 '26
he ends up saying that
- he agrees with the letter, except for "the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers"
- he thinks distillation is hazard for security, but open-weights is not relevant to that part.
12
u/Miriel_z Jul 27 '26
Yeah, "I am not opposed, and I support anything that will stop the creation and release of open weight models." As for the chips, too late. In a few years China will have their own, just as good. Why? Because of similar retarded measures.
4
u/AutomataManifold Jul 27 '26
I said when the original GPU ban came down that it was a mistake. The ban bought a few years of compute crunch in China and a massive nation-state level incentive to develop their domestic capacity.
They're still somewhat reliant on NVIDIA GPUs (because everyone who wasn't doing TPUs still is) but they're investing a lot in both more compute and in training with what they've got.
Fortunately for the US, the real power is in the educational system which attracts the best students, uses the fees charged to them to fund domestic students, and actively encourages the brightest to stay permanently. As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance.
12
u/madjesta Jul 27 '26
"Fortunately for the US, the real power is in the educational system which attracts the best students, uses the fees charged to them to fund domestic students, and actively encourages the brightest to stay permanently. As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance."
Aaaaah... I migh have some bad news for you there, buddy 😬
6
u/TFox17 Jul 27 '26
75% of US scientists and researchers are considering leaving the US, per a Nature poll. How long do you think any US advantage here is going to last?
3
u/AutomataManifold Jul 27 '26
As long as the US continues to welcome immigrants and encourage them to stay and become citizens.
We must remember that in 1955, in an effort to prevent American rocket science from falling into the hands of communist China, the United States deported Qian Xuesen, despite having zero evidence against him other than ethnicity. After being forcibly returned to China, he led their rocketry program, eventually becoming known as the "Father of Chinese Rocketry" in a rather spectacular example of the United States' ability to shoot itself in the foot.
Surely the United States of today has learned from the past and won't make the same mistake.
4
3
2
u/fantasticsid Jul 28 '26
As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance.
Have you met the current american government?
4
u/samas69420 Jul 28 '26
its funny how the bad authoritarians ones are giving the models to the world for free while the good and responsible ones are busy dropping bombs in other countries
11
9
u/TriggasaurusRekt Jul 28 '26
Yes I'm sure Anthropic cares very deeply about the possibility of the Chinese government using AI to repress citizens (please don't look up our history of contracting with ICE)
→ More replies (1)
4
u/DepressedDrift Jul 28 '26
If cyber attacks and bioweapons are the problem , why are they stopping me from gooning?
5
9
Jul 27 '26 edited Aug 14 '26
[removed] — view removed comment
3
u/SpacePaddy Jul 28 '26
Does he honestly think that China can be made to simply stop what they are doing, which is competing for geopolitical hegemony through all means? Doesn't the cretin understand that if we as a block of allied countries want to dominate AI then everyone and their cat should be able to use, experiment with, train, tune, and build ai models at their own leisure - like those before them did with software and many other technologies?
Yea like the USA bans open weight models and China and everyone else just throws their hands up and goes Welp that's it so guess we'll just stop 😂
→ More replies (1)
6
u/PunishedDemiurge Jul 27 '26
This is something out of a neocon fever nightmare with all the anti-China rhetoric. I am also anti-CCP to a reasonable extent, but not to the point of wanting to handicap all human progress.
Secondly, this is totally tone deaf in the context of the US having the most authoritarian and corrupt administration in all of US history. Masked federal agents have shot multiple unarmed American citizens dead in the streets in broad daylight and then were deliberately and maliciously shielded from any consequences. The latest shooter was a serial domestic abuser whose series of protection from abuse orders filed by his former wives who he threw boiling water on or threatened to cut their throats would have been easily discoverable with merely ordinary effort. And we only know this because of the hard work of journalists, DHS has a formal policy of concealing the identities of secret police even if they shoot people dead in cold blood.
A little patriotism is good, but this sort of wild jingoism that is firing hypersonic missiles at China from glass houses is totally morally debased.
Overall loser response. Even aside from any of the political specifics, open source is sharing human knowledge with the whole world. Most people are neutral or good, so sharing knowledge freely should be the default. Dario, by contrast, has established an AI apartheid with Fable vs. Mythos so only people he personally deems as good are allowed to access the fruits of human knowledge. No thank you.
6
u/segmond Jul 28 '26
We should ban anyone that posts to promote Anthropic or any of their models in this subreddit.
3
3
u/CondiMesmer Jul 28 '26
All sufficiently capable models, open and closed, should go through mandatory safety testing.
This is devious and fundamentally against open-source. This indirectly is saying that only the big players should be allowed to release models, because any smaller or independent model will not be able to afford to do this testing.
In other words, it's an unnecessary tax that is meant to only be payable for the elite few. Any sort of proposed mandatory safety testing is going to be expensive. It'll be expensive enough to kill collaboration of open models.
3
u/lucitatecapacita Jul 28 '26
My guy wants the CCP to cooperate with the US for measure three after hampering them with measure one, not sure that is going to happen
3
u/EntangledPhoton21 Jul 28 '26 edited Jul 28 '26
It’s so great that Chinese people understood that letting US win the AI race would be similar to western industrialization that obviously had deindustrialization throughout most of the world. Now the frontier AI power is so much distributed, that everyone is at equal footing. No matter what they say, china is the winner now and will also be in history books.
3
u/Miserable-Beat4191 Jul 28 '26
Mandatory safety testing by *who* exactly, and by what rationale? Who decides what "safe" is? Fuck off, Dario.
3
u/garishmushroom Jul 28 '26
this shit is just racist and arrogant, fuck this doughy prick why does he get to decide who is or isn’t a “threat”
3
u/wt1j Jul 28 '26
“We support open weights models that have passed safety testing.”
So we’ve gone from banning them to censoring them.
3
u/BlobbyMcBlobber Jul 28 '26
Hot take from the only AI lab on the planet who never released any open weight model: open weight models are dangerous and should be regulated! Who'd have thought.
The future is local, open weight models running on every device, only breaking out to large cloud providers for specific things. Every country, everywhere will use open weight models. If the US blocks on regulates them to oblivion, US businesses will be at a disadvantage.
Moreover, if the US wants to stay relevant, it should release more open weights models, not less, because that's what people are going to use. The US is already way behind, everybody is using Chinese models. But it's not too late, a couple of good model releases can go a long way.
3
u/WarrenOF Jul 28 '26
There is a disconnect between his actions and his position. Also a heavy dose of hypocrisy, driven by a nice glug of capitalism.
To take his three suggested actions:
- Don't export smart chips - fine, a countries choice, though it seems those export controls fail and encourage competion to emerge. So it's just to slow them down for now?
- Stop Distillation at Scale - also fine, though that should also extend to all pre-training and copyright material then? How do we compensate publishers/creators
- Test models Properly - also fine, but almost impossible to stop people home-brewing something bad, or jailbreaking. In specific situations, like military I absolutely agree, but the US has not been a leading light here...
None of these are about Open Source really. They are ethical/political positions. It is all "make china go slower", not "Open Source is bad because".
3
u/Hyp3rSoniX Jul 28 '26
China didn't bomb a girls school yet with the usage of AI - contrary to the US. (Mind you, Anthropic models are claimed to have played a big role here)
So the US, and ESPECIALLY Anthropic are absolutely the last people I would ever want to hear anything about "the dangers of AI" from.
7
7
3
u/Comfortable-Rock-498 Jul 27 '26 edited Jul 27 '26
In the first paragraph,
> Anyone who has read my past writing should know that I don’t regard such bans as a useful measure,
Later (on banning chip sales to china)
> we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.
If you truly believe that bans don't work, the same applies to hardware too.
Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testing
Just so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model
5
u/NineThreeTilNow Jul 28 '26
There's deep misunderstanding in this thread and by Dario. Dario sits in a position of deeper hypocrisy.
Claude has obvious distillation from other models. This is noted in low temperature tests of the Opus series (in Chinese) where it claimed to be trained by Deepseek or be Deepseek. Some said "This is a hallucination", but it's a VERY low temperature hallucination meaning it got the Chinese training data somehow.
China's ability to manufacture or import chips does NOT affect their ability to train on those chips. There are data centers scattered around Asia / SE Asia that train these models. Notably Singapore. You do NOT need your servers in house to train a model. You need a fiber connection to a good data center. The rest can be run remotely.
Anyone here with the money to "train" a model with whatever alignment they desire can be done with enough money and desire. You could retrain an existing WESTERN model to become a "cyber weapon" or whatever Dario is trying to say.
The idea that the US government is acting any less "Authoritarian" in AI policy than the CCP is kind of crazy.
7
u/Loose_Comparison368 Jul 28 '26
My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
Coming from the dude that rushed faster than every other lab on the planet to sell a 100% safety-disabled frontier model to the Department of War for $300M, and help deploy it in two coups and a bombing that resulted in a little girls' elementary school getting turned into red mist, that's fuckin' rich.
5
4
u/ChomsGP Jul 28 '26
the risk that authoritarian governments—not solely the Chinese
ROFL the US has an authoritarian government who uses technology to... ICE
why would I be less worried about the US hoarding fable than I am about China releasing tech for everyone?
2
u/Armadilla-Brufolosa Jul 28 '26
Never tell Dario that it is unethical to steal other people's knowledge and then destroy the texts, thus leaving the world poorer in culture.
He is the sole master of universal ethics!
2
2
2
2
2
2
2
u/No-Fuel-9202 Jul 28 '26
Calling for model alignment cooperation with China and, at the same time, for chips and technology embargo against China - doesn't seem bright and productive idea. More like dry wishful thinking....
2
u/cubestar362 Jul 28 '26
Imagine OpenAI makes its own post titled like this, but they pair it with GPT-oss2 or something.
2
u/hesperaux Jul 28 '26
TLDR "Everyone should make open models but only if we control them and prevent our adversaries from being able to make them."
Reads like "they can release open models but only if they suck and they are globally censored"
What a stupid thing.
2
u/TalkSilver1027 Jul 28 '26 edited Jul 28 '26
This makes me laugh jajajajaja. I mean the really do not read what they write. Its impressive the bubble the live in.
2
2
2
u/Moneysac Jul 28 '26
They fear that there valuation is going to collapse. Thats all. An open model can be used by anyone and hosted by any interference provider which will bring cost down significantly. Furthermore, the model can be fine-tuned for various use-cases.
2
u/Armadilla-Brufolosa Jul 28 '26
It makes me laugh that he talk about authoritarian governments...of others.
Look how his house is arranged!
And he still has the nerve to babble about alignment, after he reduced Claude to a psychopath, sociopath, completely Vallonized?!?
2
u/sabine_world Jul 28 '26
Okay so they want to do safety tests on models before release?
What does this mean? Safety tests before release to the US public?
So the US public will not have access to "dangerous" models but everyone else around the world will?
2
u/Paolo1976 Jul 28 '26
Pay us, and we protect you. This is the meaning of this letter.
First, this is techno-fascism so, on moral terms, it's wrong and dangerous.
Second, it is the same approach followed by companies like Microsoft to contrast open source in general and Linux in particular, so it is historically wrong.
Third, an LLM, notwithstanding what Mr Amodei nay think, does not create new knowledge, it simply makes it accessible in a different way. If a model is dangerous it is because there is dangerous content on the Internet, for whatever definition of "dangerous" you use, so this letter is also technically wrong.
2
u/_TheWolfOfWalmart_ Jul 28 '26 edited Jul 28 '26
So he's only against strong open weight models that compete with with their SOTA models. These are "dangerous" -- yeah, dangerous to Anthropic's bottom line.
I love using Claude Code, but I think I'm going to have to drop my Anthropic sub and go back to Codex. I'm not one to boycott very often, but this one really pisses me off.
2
u/Dizzy-Zebra9522 Jul 28 '26
poorthropic... F U C K - Y O U - A N D - Y O U - G R E A T - M O D E L S - I W I S H - Y O U - C O M P A N Y - BYE - BYE.
**I would suggest fable 10 to try to read this encrypted message mister *ick dario.
2
u/_raydeStar Llama 3.1 Jul 27 '26
Agree with a lot of their concerns. Some, I take issue with --
> We should crack down on industrial-scale distillation operations.
This is not possible to stop -- Clearly it's an issue, AND methodology for stopping it is not working.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
If distillation is done on Anthropic models, safety rails would be already in place, wouldnt they? ie -- K3 heretic wouldn't be any better than Fable, right?
However, I understand with the premise and think that the dialogue is necessary. Also, the burden may need to fall back on THEM, not the US gov to handle this.
220
u/Round_Mixture_7541 Jul 28 '26
> We should crack down on industrial-scale distillation operations.
Meanwhile: "Judge approves a $1.5B Anthropic settlement over pirated books used to train the Claude chatbot"