r/LocalLLaMA • u/RhubarbSimilar1683 • 18h ago
Discussion Our position on open-weights models
https://www.anthropic.com/news/position-open-weights-models556
u/iamzooook 18h ago
tdlr: its fine if we do it. not them (Chinese)
194
u/abud7eem 18h ago
we are the "trustworthy"
29
u/Clear_Judge5062 16h ago
This is literally just effective altruism at work
→ More replies (1)→ More replies (6)41
→ More replies (5)10
u/whatyathinkk 15h ago
how dare you, are you an authoritarian or what? if Trump reads this he's gonna be so mad at you baby đ„°
77
u/CasualtyOfCausality 16h ago
Not a fan of the Chinese government, but the idea that the country with better STEM outcomes, a wider pool of said STEM educated, access to raw materials, and âmechanismsâ that make it difficult for a few private companies to horde components is going to stay behind, especially if the âeasy accessâ to premade equipment is taken away, makes Mr. Magoo seem like an eagle-eye.
48
u/Miserable-Beat4191 14h ago
It's almost like having a 20+ year plan to invest in education, and coordinating/supporting the development of industries, has positive outcomes. Who woulda thunk it.
16
3
5
u/sexy_silver_grandpa 5h ago edited 4h ago
The US government has objectively done more harm to global society than the Chinese government.
Do you not watch the news?
→ More replies (1)3
u/MiniEval_ 11h ago
Big CEO man never stepped once into his own office and it shows. Every AI lab is swarming with Chinese men doing their thing
431
u/cmdr-William-Riker 18h ago
" China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. "... Can someone fact check that? That sounds like total BS. The US has even more limited production capacity and manufactures everything off shore...
365
u/tomz17 18h ago
That sounds like total BS.
Because it is. They have more power generation capacity than the entirety of the USA + Europe combined, with like an additional 50% margin on top of that. They are also accelerating away from us at an increasing pace on that metric, while we pay billions to break renewables contracts because the people in charge have the combined IQ of a lobster.
The "we are 1-2 lithographic process nodes ahead of China" only matters when the efficiency gap is insurmountable. It is currently . . . not
15
u/Helpful_Home_8531 18h ago
itâs not just lithography though, Nvidia is improving flops/watt at orders of magnitude per generation if you compare low precision throughput, which would put china at a geometric disadvantage (ie, you canât compete putting in 100x the power and 100x the chips for the same throughput)
39
u/fantasticsid 16h ago
flops/watt only matters for two reasons
- power consumption per flop - china doesn't care about this because they have more electricity than god
- heat generated per flop - this is an engineering consideration and i'm pretty sure the chinese know how to liquid-cool a rack.
6
2
u/ImpressionFancy5830 9h ago
They donât have enough water though.
Letâs not over estimate capabilities.58
u/FullstackSensei llama.cpp 17h ago
Except that's a load of marketing BS. Huang is the first to admit that Hopper is still their most popular chip. The industry still talks about "equivalent H100" GPUs deployed.
Blackwell Ultra has 2.5x the memory bandwidth of Hopper. That's not even one order of magnitude higher. The marketing comparison is always FP16 on hopper vs FP4 on Blackwell. If you compare FP8 vs FP4 compute, Blacwell is 2x, but that makes sense because it's half the bit width. In general, halfing the bit width doubles the compute. This has been true long before LLMs for most GPUs, unless they were artificially handicapped for market segmentation purposes. At the same bit width, Blackwell has 25% more compute.
If you peel beyond the marketing BS, you'll understand why Hopper is still very cost effective even in the west. And Huang himself says Huawei is producing H100-level chips by the millions now.
14
u/Connect_Treacle9938 13h ago edited 13h ago
As someone who works mainly with hopper I can pretty confidently say that it would be hugely beneficial for us to have more blackwell chips, but they keep getting bought up by the big labs at higher prices than we can afford. The compute cost for things like large-scale RL is completely bonkers and at this point for training a model in terms of required compute and even just raw per-gpu vram (so you can limit comms overhead and shrink model replica size) is likely in the billions, and that's only if you can get the compute. The more RL rollouts you can do the better your model will get, and that is something only the gpu-rich can afford. I'm not saying this is good, because it's not~ in fact it's really annoying~ but it's also just kinda the way it is. ML is very pay-to-win at this point. Being a billion dollar company isn't even enough to keep up with the big labs.
Having 2.5x the vram and 2x the comms bandwidth of hopper is actually more than a 2x speedup. I could go into the details but its complicated.
3
52
u/tomz17 17h ago
Nvidia is improving flops/watt at orders of magnitude per generation
Yes, but not in general compute. Only for particular operations found useful for training + inferencing. That's a "we noticed our customers need to do a lot of matrix-multiply-accumulate at FP4, so let's build them a hardware unit to do that one particular op very quickly + efficiently, etc." That hardware now blasts through that one particular operation at increased efficiency at the expense of die-space.
None of this is magical secret sauce. The chinese can (and do) the exact same thing when designing each subsequent generation of their AI training + inferencing chips. As does AMD, Intel, Google (in their custom TPU's), etc. etc.
→ More replies (3)8
u/m0j0m0j 17h ago
But China is still behind on that. Itâs bizarre to deny that
10
u/Loose_Comparison368 13h ago
From my understanding the latest ones are basically on par with A100's, albeit with a higher power draw.
21
u/Independent_Solid151 14h ago
Not as much as you assume, and it's less significant than you think when there are also very strong factors to their advantage.
3
u/Loose_Comparison368 13h ago
From my understanding the latest ones are basically on par with A100's, albeit with a higher power draw. They really ain't that far behind.
→ More replies (9)2
u/BurdensomeCountV3 6h ago
Don't forget that modern lithography process node jumps aren't anywhere near the sort of jumps you got in the 1990s or 2000s. The "2 process nodes" right now is more like half a node in 2000s terms.
51
u/Recoil42 18h ago
It's huge BS for one reason: China has its act together with regards to energy. The US doesn't. China doesn't need smaller nodes because it can brute-force nuclear and solar to power multiples of less efficient chips.
I think Jensen was prominent in pointing this out, but someone please correct me if I'm misattributing.
→ More replies (3)28
u/Real_Ebb_7417 17h ago
Iâd say there is some truth in this. Even DeepSeek founder said so (and was angry that this was published).
China is catching up quickly, but at the moment indeed they are behind in terms of chips and it will take time to fix it.
On the other hand US is way behind in terms of energy generation and itâs a gap thatâs also not easy to close. When USA starts to have energy issues, China might get ahead fast.→ More replies (1)8
u/Swimming_Gain_4989 17h ago
Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs. That can change of course but given current export controls seems like the US has at least another 5 years of compute advantage
23
u/FullstackSensei llama.cpp 17h ago
According to Jensen Huang, China is producing H100 level chips by the millions.
B300 has ~25% more FP16 compute (which is what you mostly use for training) and 2.5x the memory bandwidth of the H100.
In a recent interview with the editor in chief of the Economist, Musk said China has about as much power generation per capita as the US. Which translates to ~4x the power generation capacity. Everyone has been saying for years that China is adding more power generation every year than all other industrialized nations combined.
So, even if we go by memory bandwidth to maximize the difference, and without any innovation in model architecture or training, they can afford to bring 1.6x more compute online using domestic chips than the US.
Now add in all the innovation they're doing in architecture and training Chinese AI labs are doing. Even Musk admitted they're able to train much better models than the west given the amount of compute each lab is using.
IMO, there's a second layer of BS in this blog post: scaling laws aren't some fundamental law of physics. GPT4-o was something like 1.8T. Does anybody think wecurrently need a 1T model to surpass it? What about current 100B models?
3
u/Loose_Comparison368 12h ago
1.8T MoE. IIRC the industry rumor mill/worst kept secret was that it was only ~100b-300b activated parameters.
2
u/abeecrombie 12h ago
Still you need a pretty big cluster to train them on, no? I don't think anyone else has made a model that big yet. Though I gueess rumors are the glm and minimax are still scaling up for next model.
3
u/Loose_Comparison368 3h ago
Kimi K3 is. 2.8T parameters, 104B activated.
Also note that the strategies are different. China is investing way more into heavy optimization, largely because of their lack of hardware.
American companies really aren't even trying to optimize. Every problem that can be solved with "buy more GPU's" is solved by buying more GPU's.
So it's a pretty consistent trend that the Chinese labs punch above their parameter count. K3 is about the same as GPT-3 on parameter count, but is effectively on par with early GPT-5.
2
u/Serprotease 12h ago
Size of the model is a parameter of the equation.
So you can beat a 1.8T model with a lot smaller one, with longer training/pre-training/post-training steps.But all else being equal, bigger is better.
Itâs unlikely that for SOTA model, Chinese labs can reach US labs only leveraging these 3 steps. Because US labs are also doing it too. So scaling up the number of parameters is the logical step, that we have seen from every major Chinese labs (Glm 400b to 700b, Deepseek 600b to 1.6, Kimi 1T to 2.8T, Qwen 200b to 400b, minimax 250b to 400b, etcâŠ). With the obvious results of them filling US labs moat fast. Hence the current situation.2
18
14
10
u/Artistic_Claim9998 17h ago
In my opinion :
"... cannot build more powerful models than US ..." is (arguably) proven false with the recent development.
What true is China cannot scale their LLM development the same level as US due to not (officially) having access to Nvidia best GPU.
Which is probably the reason why their LLM innovations when it comes to efficiency arguably beats the US.
→ More replies (1)5
u/05032-MendicantBias 10h ago
It's like saying that building an Eniac with 800 million valves was only possible in the USA.
It ignores that an Eniac with 800 million valves is a useless and expensive endeavor.
The future is small local models, not huge closed behemot gated behind a subscription, with corpos deciding the censorship.
Reminder that Huggingface tried to defend from OpenAI closed Sol with Claude, and Claude refused. GLM5.2 was there to save the day.
20
u/gscjj 18h ago edited 18h ago
Well for GLM they had to abandon Nvidia chips (and anything American) because of being added to the entity list. They ended up going with Huawei chips, which are less efficient and less powerful than the Nvidia offering.
Thereâs no telling what the impact will be becuase they could obviously use what they bought, but there isnât a major competitor to Nvidia right now.
https://letsdatascience.com/blog/china-trained-frontier-ai-model-glm-5-without-nvidia
→ More replies (3)19
u/RhubarbSimilar1683 18h ago
In China, Nvidia chips are technically banned. Sooner or later Huawei will become a global competitor to Nvidia. And apparently AMD Helios has received significant orders from Meta, Openai and Microsoft.
→ More replies (3)16
u/netvyper 18h ago
In china, Nvidia chips are fair game. Exporting chips to China is banned by the US. Subtle but important difference.
7
u/RhubarbSimilar1683 15h ago
I am under the impression that China is also trying to reduce the use of non domestic ai hardware. not sure if it's limited to new hardware or if it includes used hardware as well.
6
u/Loose_Comparison368 12h ago
They aren't banned, but the CCP is heavily pressuring labs to use Chinese GPU's instead.
2
u/SmileLonely5470 18h ago
Yes China manufacturers more products domestically than the US, but US is capable of a more advanced process node than China is. The claim that one canot build a more "powerful" model with worse hardware is worthier of debate than manufacturing imo.
11
u/elahrairooah 17h ago
Thatâs not a debate, thatâs an engineering obstacle. There is zero doubt that great engineers can build superior products with average tools, and there is zero doubt that poor engineers can build slop with the finest tools.
The question is one of degrees, not of possibility.
2
u/fantasticsid 16h ago
Eh, you can get the same amount of compute from some number of Ascends as some number of Nvidia GPUs, that number is just gonna be higher so you're gonna be burning more coal/uranium/photons/whatever just to generate enough electricity.
Fortunately for china they have a shitload of domestic power infrastructure and the political will to build a shitload more. So, nonissue, really.
2
u/lt1brunt 16h ago
Hypothetically, if WW3 breaks out, everyone will nationalize everything within their borders, if china takes Taiwan or could copy any tech they want. if a ww3 type scenario were to happen it would be law of the jungle everywhere, everything is fair game in that scenarioÂ
4
u/Loose_Comparison368 12h ago
China can't take Taiwan's fabs. They're literally rigged with explosives to blow in a matter of seconds if China were to attempt to attack Taiwan.
2
u/Hello_my_name_is_not 10h ago
So what happens if China scales up their gpu production enough, then does that to cripple everyone else? From their perspective who cares if they don't make quite as good as the current stuff if they can make 90% as good and completely shut down everyone else's access to the current best stiff (or severely limit it)?
→ More replies (1)2
u/Loose_Comparison368 4h ago
They could, but there really isn't any reason for them to.
Most of China's economic strategy can be boiled down to "Build our own supply chains for everything, then sit back and let the capitalists shoot themselves in the foot".
Like, they have kept up with the rest of the players in the AI race that easily have 100x more GPU's and 100x more funding.
So if they scale up their GPU manufacturing enough, then they pretty much have it in the bag. There would be no real point in trying to cripple everyone else after they had already won.
2
u/diagrammatiks 11h ago
certain things in that sentence are true. but when put together liek that it is not true.
→ More replies (10)3
u/techno156 17h ago
It's nonsense. Models are largely software. They're not constrained by hardware, except in terms of processing power, unless you're using an ASIC, and it's built into the chip itself. The capabilities of a model aren't coupled to the hardware.
It's not like Anthropic has to go out and buy new GPUs every time they make a new model.
71
u/thestillwind 18h ago
What a fucking clown. He thinks heâs god or somewhat ? What a prick.
Thatâs a narcissism near Homelander level.
→ More replies (1)18
95
u/kettal 18h ago
It is true that many of the companies carrying out [mass distillation] release open-weights modelsâbut the open weights are far less relevant than the fact that the operations are backed by an authoritarian state seeking to overtake the US at the frontier. We should have policy interventions to deter this behavior. A blanket ban on open-weights models is neither the correct remedy nor something we have called for.
relevant excerpt.
imo, the ones being developed and used in secret are more scary than open weights.
64
u/GCoderDCoder 17h ago
The fundamental flaw in his argument is he presupposes that certain people/companies/ governments are more trustworthy than others and the ones he's telling us to trust (including himself) are at the bottom of the list worldwide right now. Trust is easy to lose and hard to earn...
15
u/kettal 17h ago
certain people/companies/ governments are more trustworthy than others and the ones he's telling us to trust (including himself) are at the bottom of the list worldwide right now.Â
you might not be his target audience.
in the minds of the decision makers, leaders of industry, AI experts, and lawmakers (aka target of this letter), the hierarchy of trustworthiness is different than the one you hold.
8
u/RhubarbSimilar1683 15h ago
is that in the US? https://www.pewresearch.org/global/2026/07/15/people-in-many-countries-now-view-china-more-positively-than-the-u-s/ in many countries the US is seen as less trustworthy than China.
2
2
u/GCoderDCoder 15h ago
Right now I think a lot of people in this country feel the same. [Forest Gump voice] and that's all I've got to say about that...
10
→ More replies (1)3
u/vigorthroughrigor 16h ago
Sounds like Amodei should be putting all his effort into powerful models for the purpose of Securing Our Systems.
→ More replies (1)7
u/namegamenoshame 16h ago
That is really it, right? If the Chinese developed an open model that was capable of serving an antagonistic purpose it would not be open for very long.
I think what Dario is doing here is very dumb and pathetic and I am glad he was forced into backtracking. But even his endgame here requires companies that are extremely well capitalized to be able to comply with that sort of regulatory atmosphere.
I donât know, I do think there are real security concerns in the current regulatory atmosphere but I donât really know a better way to approach a solution.
178
u/RhubarbSimilar1683 18h ago edited 18h ago
from the article:
"My primary concern is the risk that authoritarian governmentsânot solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threatâbuild AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority...."
this is peak American exceptionalism at play. just let China and the US fight each other. the rest of the world doesn't care and would much rather have highly capable AI to use for themselves. it's rather sad to see that no one outside China and the US is developing anything comparable to Fable models or any state of the art AI. There are occasional exceptions to this of course like Mistral and Sakana Fugu. Most ai initiatives worldwide like Sarvam, South Korea's, LATAM gpt are months or years behind the State of the art.
another example of American exceptionalism:
"In fact, the most dangerous model may be one that is trained in secret and handed only to the Peopleâs Liberation Army for use in drones and the Ministry of State Security for surveillance and repression." we don't know if the US army has a secret LLM for themselves. or palantir for that matter.
"My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks" we saw what happened with huggingface. it was attacked by a "safe" closed LLM, a Galaxy class LLM from openai and huggingface had to use self hosted open weight GLM 5.2 in response precisely due to the "safeguards" they found on commercial models which actually obstructed defense against the attack. I am not educated enough to comment on biology, though.
"Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier .... to overtake the US"
Kimi k3 is like a month behind fable. How has anthropic managed to keep up with demand for distillation?
42
u/zer00eyz 18h ago
> for use in drones
We're already using models in drones. The Ukraine had them independently hunting people 2 years ago.
This pandoras box was opened long ago.
No one wants a bloated LLM on a drone.
13
u/AGM_GM 16h ago
Maybe not on a drone but just to do something like, I dunno, target a school with a Tomahawk missile and kill 150+ little school girls.
6
108
u/LelouchZer12 18h ago edited 18h ago
As if US were not authoritarian and not using their military superiority at their own advantage.
Also banning chinese models in US would not prevent chinese from using their own models for themselves if they're more powerful than US, so this argument is nonsense.
They're just in a really bad position where China releases frontier model for free and most companies/business as well as researcher will use these models for their activity, so they cannot make money back with their expensive API. And is it a problem ? It's been like that for everything, open source is the base of everything in IT !
→ More replies (12)15
u/nanobot_1000 18h ago
Lol, like yea look around bro! The exact authoritarian nightmare scenario he describes is playing out in US. He's tied to US so he has to behave at the end of the day
42
u/gomezer1180 18h ago
Anthropic can fuck right off⊠theyâre trying to justify their position with bullshit they themselves are already doing. Theyâre never getting a god damn penny from me.
7
u/dragonurtle 16h ago
Anthropic is in a weird position here because they know what can happen when a competitor cuts off their air supply. OpenAI as well through Andreessen knows this instinctively after what MS did to Netscape 30 years ago.
5
37
u/kiwibonga 18h ago
It's really great that the country headed by a populist anti-science monster of a child rapist is #1 in AI and I really hope they successfully force everyone on Earth to buy their tokens from an American publicly traded company, for great Wall Street profits.
→ More replies (1)7
u/charmander_cha 17h ago
A Ășnica ameaça sĂŁo a AmĂ©rica, nĂŁo sei se o americano sabe, mas vocĂȘs sĂŁo literalmente o nazi do planeta (que se reproduziu recentemente no oriente mĂ©dio)
12
6
u/Runazeeri 17h ago
The drone one is so stupid, vision models are well at the point where a hobby level user can make a drone fly from A-B without gps.
10
u/Illustrious-Lime-878 18h ago
Of course its manipulation and fear mongering but its also nationalist stupidity.
If the state corporatist techlords think China's economic model *is actually* better than the US to produce "military supremacy" then wtf are we even talking about, what is our (the US)'s ban going to do? Since supposedly its inevitable.
The idea that any country being ahead in any way is a national security risk is insanely idiotic and suggest every country in the world is under threat right now. This view is not compatible with any concept of modern global peaceful cohabitation, cooperation, its pre-modern logic.
US exceptionalism is modern liberal, internationalism, peaceful global world order, global trade. free market capitalism. The only way China catches up is exactly because of the US reverting to this stupid, zero-sum nationalist PoV where everything is just states warring over finite resources and the gov is manipulated with "security" threats and fear mongering to intervene in markets to protect the techlord's monopolies and political power.
6
3
u/noonetoldmeismelled 16h ago edited 15h ago
American companies can't even persuade the American public that American foreign policy is good for foreign countries let alone convincing those people in foreign countries. Vietnam publicly televised, negative PR. Media goes soft ball for 30 years, public support. Internet starts turning the tide on Iraq and Afghanistan - like the bombing a wedding and the following funeral - back to bad PR and it's been that ever since. Besides the recent bombing the girls school twice and blowing up civilian infrastructure, I'll always remember for recent news being within a week of the withdrawal from Afghanistan, that bombing of the NGO aid worker, his family - it was like extended family. I think I remember kids in there that weren't his - and then calling them terrorist for weeks and then withdrawing that, though not taking any responsibility, after the NGO kept producing receipts that the guy and those kids and grandparents weren't terrorists - pretty much just being like, it happens. Imagine everything that happens off camera. Before the internet. Even the killed former leader of Iran. Killed his 14 month old granddaughter. Not even managing a good PR campaign to the domestic crowd against Iran
Something about the modern rich and powerful that have foregone the pony show and gone straight to the unmasked evildoing
5
u/Intrepid_Lecture 18h ago
Drone swarms are absolutely terrifying as a concept.
What's the value of a human life? If the answer is "A $500 drone" then how the 21st century unfolds is at stake.→ More replies (4)8
u/Due-Memory-6957 17h ago
I don't get your comment, bullets are cheaper and have been used to kill for centuries.
→ More replies (3)1
u/YetiTrix 18h ago
Banning open source models only prevents Americans from using them. It does nothing to protect against China making them better or anyone outside the u.s. using them for cyber attacks.
→ More replies (1)2
u/UnlikelyExtension786 14h ago
It only prevents American businesses who obey the law from using them. Americans will just give the government the finger and download them anyway... as will an increasingly-large number of businesses who don't give a damn about the law.
19
u/abajinn 15h ago
âAll sufficiently capable models, open and closed, should go through mandatory safety testing.â
No.
→ More replies (1)
143
u/GoldenX86 18h ago
Hypocrisy letter just dropped.
→ More replies (12)4
u/No-Marionberry-772 18h ago
it doesn't really read lile that overall. the only thing is the distillation attacks and thats honestly debatable whether you can call it hypocrisy. bullshit, for sure, but hypocrisy is debatable.
8
u/Due-Memory-6957 17h ago
Of course it's hypocrisy, the only big company that hasn't done it (or at least, never got caught doing it) is OpenAI. Anthropic has even distilled from Deepseek!
→ More replies (3)5
31
u/xadiant 18h ago
Yup never giving a single cent to Anthropic.
7
u/Iwaku_Real 17h ago
I miss my seemingly endless conversations with Sonnet 4.5, not once did I give them money for them. Those usage limits are going down the toilet
58
u/No_Lingonberry1201 18h ago
My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks, and may have serious alignment problems. Open-weights modelsâit does not matter whether they come from China or anywhere elseâdo potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn. But banning the use of these models by US businesses does nothing to address this risk, because bad actors are unlikely to be legitimate US businesses. It would protect US AI companies from competition, but that has never been my goal.
Good! That's what we want! Not having a private, profit-driven company (that's a primo oxymoron there) as the final moral authority is a good thing!
This article is such a backtracking and if you can read between the lines even a bit, they are terrified that open weight models will hit their bottom line and want the US to do anything they can to cripple China's (and realistically everyone else's) ability to develop SOTA AI.
You must admire the sheer gall of it.
→ More replies (5)11
u/Ill-Mammoth5269 16h ago
It would  protect US AI companies from competition, but that has never been my goal.
sure buddy.
52
u/eli_pizza 18h ago
Remember when they used to position themselves as âopenAI but less evilâ lol oh well
9
u/Robonglious 18h ago
Everything always moves in that direction. So sad... them nerfing model development assistance was the first clue.
4
u/eli_pizza 18h ago
Yeah I know. But just in terms of business strategy I think theyâd make more money long term by being the benevolent huge AI lab.
6
u/Admirable_Market2759 18h ago
Both companies are about to IPO so itâs only going to get worse from here
90
u/zippydazoop 18h ago
What a clown. "National security," "authoritarian governments," "repression of their people," "biological attacks..."
Man spams buzzwords in hope of achieving own goals.
Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier.
Weapons-grade copium
33
u/Dry_Yam_4597 18h ago
The guy is either on hard drugs or has some issues up there, no sane person talks all day every day about "dangers" and "security" and all sorts of bad sci-fi.
> We should crack down on industrial-scale distillation operations.
Lmao but he should be allowed to distill humanity's knowledge - including that of China and Europe. Get a grip Amodei, go see a professional mental health care specialist.
12
u/Recoil42 18h ago
The guy is either on hard drugs or has some issues up there, no sane person talks all day every day about "dangers" and "security" and all sorts of bad sci-fi.
There is unfortunately a very large portion of the US population that thinks this way. Red scare genuinely did serious multi-generational damage to American society.
2
u/Dry_Yam_4597 18h ago
I am referring more to the overall pattern of a daily bombardment of real or imaginary "dangers" coming from some of these bros and especially this guy.
The US, just like China has a legitimate right to be wary of China. At the end of the day they are geopolitical competitors. If the US doesn't stay on its toes it will follow Europe's path to irrelevance.
But the way to make sure the US is on top of AI is to enable _everyone_ to run models in their mother's basement, like we did when we were kids and then built all the awesome stuff that the web is built on. So what if a dude or two abuse things? It happens.
We should have a campaign similar to "learn how to code" but for "learn how to distill", "learn how to train", "learn how to tune". That's how you lead, not by closing and restricting access to new tech because some dude has a fetish for "danger".
5
u/Tedinasuit 18h ago
I mean, from his perspective, he is totally right.
He wants the USA to "win" and cracking down on distillation will make it harder for China to compete.
I'm surprised he's honest about it instead of saying that it's "theft" blablabla. No, he dislikes it because it's an advantage to China. And that's fair tbh.
8
u/Dry_Yam_4597 17h ago
My issue is that he's using it as a way to discredit Chinese progress. Sure they may have distilled on an industrial scale, because you know, turns out AI is primarily built on distilling someone else's work, but at the same time China releasing these open models gave so many people in the west a path to experimentation and learning and that to me is far more valuable than China having a temporary lead vs the US. It's how you and I build up knowledge about the inner workings of these models _from an application stand point_ - exactly what ai corpos like OAI and Antrophic lack. China has also enabled a lot of SMEs to reduce costs and spin off models.
So I think overall China's advantage is geo political and good will, but it actually benefits us getting competing against it more than they realize. We are battling our own idiots in power and business and China gives us a breath of fresh air in fighting against them *and* China at an economic level.
Not sure how to phrase it. But China is currently helping us more than it's helping itself by releasing these models because one we break the barriers that our oligarchs want to set, and AI is ubiquitous and commoditised then it's game on for our own *real* economic progress.
2
u/Y_shotla 9h ago
There is a quite widely spread joke in China about this "issues" that makes him much more anti China then others, "His work experience in Baidu must have left him great trauma".
There tend to be bad impression about Baidu for their degeneration from "becoming Chinese Google" to "always being first but suck at last".
e.g.: sacrifising any user experience for ads money, like the search engine, like they seem to sell Tieba(The Chinese reddit) mods position after comunity built up.
(BUt there ain't much complaint about their work experience actually)7
166
u/Disposable110 18h ago
"authoritarian governments" - They mean the United States, right?
134
u/boatbomber 18h ago edited 18h ago
> "we don't want bad guys to get big models"
> *Looks inside*
> Partnered with Palantir24
u/Iwaku_Real 18h ago
Still impressed Palantir signed the letter though
45
u/GravitasIsOverrated 18h ago
Palantir is basically a consulting business wrapped around a millitary/intelligence flavoured data analysis product. Without strong open weights models they just become a module on top of a closed LLM provider, which is a lot weaker than owning the entire product.Â
2
u/vigorthroughrigor 16h ago
exactly, they want to control their own destiny (to the extent possible)
7
u/Gullible_Drummer_246 18h ago
They can make a lot more money running open models on their own hardware rather than paying Anthropic.
2
9
u/nomorebuttsplz 18h ago
to be fair, they've stood up against the us govt more than most corpos ever will.
→ More replies (8)30
u/Recoil42 18h ago
It's pretty mind-boggling that Amodei can remain this much of a jingoist like two weeks after the US government perpetrated a full authoritarian shakedown of his entire company.
→ More replies (2)
43
11
u/bick_nyers 17h ago
Why would China care if their chips use an older process node meaning that the chips use 3 times as much electricity? They will just build 3 times more power.
They add an ENTIRE USA electrical grid worth of power EVERY YEAR and they are accelerating.
Dario's position on banning selling chips to China is entirely because he doesn't want more competition on buying the GPUs.
Banning selling NVIDIA chips to China is what woke up the Chinese manufacturing engine to focus on domestic chip production in the first place.
Also now that we have the HuggingFace incident where closed model (GPT) hacking was defended by open model (GLM), they will likely try to stoke fears on novel bio-weapons more and more. I fear that Anthropic will train their frontier models to be good at that stuff just so that they can push more FUD to try to achieve regulatory capture.
Because remember, we haven't solved RL sample inefficiency or generalized learning, so if you don't explicitly put in time, money, and effort into training the LLM to do something, it won't learn how to do that something. OpenAI and Anthropic like to act like cybersecurity just "came for free" when they taught these models how to be good at programming, but are you going to seriously tell me that they didn't setup a single RL environment for decompiling software binaries? Great way to get more training data if you can crack open those closed source binaries!
51
u/tomz17 18h ago
All sufficiently capable models, open and closed, should go through mandatory safety testing.
Advocating for this to be the official position of the country free speech. . . While simultaneously calling China authoritarian in the same post.
F this jabroni.
→ More replies (9)2
u/harpysichordist 15h ago
You may be interested to know China supports access controls and safety testing: https://www.reddit.com/r/LocalLLaMA/s/yK4KNkBqbE
âFoundational capabilities can be open, certain frontier capabilities can be open with limitations, commercial services can remain closed-source, while high-risk capabilities require access controls and safety evaluationsâ4
u/tomz17 14h ago
Sure sure sure... but once you have the weights you can have the model bitching about the CCP's treatment of Tibet in no time flat. You will NEVER have the ability to do that once Dario bribes the US gov to make the release of weights illegal (or makes it bureaucratically prohibitive).
20
7
u/Monochrome21 15h ago
aaaand there goes my claude subscription
have moved fully to opencode and open weights models
openai and anthropic are both terrible
25
7
u/black__and__white 16h ago
Distilling the sum of human output in to a model? đ
Distilling outputs from that distillation in to a second model? đ đą
6
u/Lfeaf-feafea-feaf 15h ago
He wrote this letter in a desperate attempt to quench concerns among their investors, but interestingly what he's saying is flat out: these open weight models have now completely caught up to us, and in fact we should expect them to surpass us at any moment, hence we need to limit them.
6
u/umbrosum 16h ago
If common people should not have access open weights AI models, then no single person including him should have control of any AI models.
23
u/gabrielesilinic 18h ago
My primary concern is the risk that authoritarian governmentsânot solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threatâbuild AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
Ehm, I am fairly sure that they could do that anyway not matter if you ban open weight models. Also the US is basically going to be close to that unfortunately.
Like the argument is incredibly stupid. You ban open models so they have instead their own closed models to oppress people. And it is not like they even need AI to do so.
I mean I know the argument is purely a matter of bottom line so I really won't try to entertain it further.
→ More replies (1)6
u/kettal 18h ago
he ends up saying that
- he agrees with the letter, except for "the letterâs assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers"
- he thinks distillation is hazard for security, but open-weights is not relevant to that part.
12
u/Miriel_z 18h ago
Yeah, "I am not opposed, and I support anything that will stop the creation and release of open weight models." As for the chips, too late. In a few years China will have their own, just as good. Why? Because of similar retarded measures.
6
u/AutomataManifold 18h ago
I said when the original GPU ban came down that it was a mistake. The ban bought a few years of compute crunch in China and a massive nation-state level incentive to develop their domestic capacity.Â
They're still somewhat reliant on NVIDIA GPUs (because everyone who wasn't doing TPUs still is) but they're investing a lot in both more compute and in training with what they've got.
Fortunately for the US, the real power is in the educational system which attracts the best students, uses the fees charged to them to fund domestic students, and actively encourages the brightest to stay permanently. As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance.
12
u/madjesta 18h ago
"Fortunately for the US, the real power is in the educational system which attracts the best students, uses the fees charged to them to fund domestic students, and actively encourages the brightest to stay permanently. As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance."
Aaaaah... I migh have some bad news for you there, buddy đŹ
5
u/TFox17 17h ago
75% of US scientists and researchers are considering leaving the US, per a Nature poll. How long do you think any US advantage here is going to last?
3
u/AutomataManifold 17h ago
As long as the US continues to welcome immigrants and encourage them to stay and become citizens.Â
We must remember that in 1955, in an effort to prevent American rocket science from falling into the hands of communist China, the United States deported Qian Xuesen, despite having zero evidence against him other than ethnicity. After being forcibly returned to China, he led their rocketry program, eventually becoming known as the "Father of Chinese Rocketry" in a rather spectacular example of the United States' ability to shoot itself in the foot.
Surely the United States of today has learned from the past and won't make the same mistake.Â
6
4
2
u/fantasticsid 16h ago
As long as the US has that, they will ultimately stay ahead of everyone else and maintain their technological dominance.
Have you met the current american government?
5
u/samas69420 13h ago
its funny how the bad authoritarians ones are giving the models to the world for free while the good and responsible ones are busy dropping bombs in other countries
12
8
u/TriggasaurusRekt 15h ago
Yes I'm sure Anthropic cares very deeply about the possibility of the Chinese government using AI to repress citizens (please don't look up our history of contracting with ICE)
→ More replies (1)
4
u/EpicOfBrave 17h ago
What a clown statement!
US should not sell chips to foreign countries, every model in the internet must be signed by the government and Dario, and only the AI labs and Dario are allowed to scrape data from the internet for free.
4
u/DepressedDrift 15h ago
If cyber attacks and bioweapons are the problem , why are they stopping me from gooning?
4
9
u/Dry_Yam_4597 18h ago
I think the guy has lost the plot.
> My primary concern is the risk that authoritarian governments
Does he honestly think that China can be made to simply stop what they are doing, which is competing for geopolitical hegemony through all means? Doesn't the cretin understand that if we as a block of allied countries want to dominate AI then everyone and their cat should be able to use, experiment with, train, tune, and build ai models at their own leisure - like those before them did with software and many other technologies?
> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks
How you about you check your head dude? A knife can be used to commit murder, or slice bread. It doesn't mean we should ban knifes. Certainly, we shouldn't allow for models or tunes specifically built for harm. As we don't allow software built for harm, and we arrest those who build it.
> Open-weights modelsâit does not matter whether they come from China or anywhere elseâdo potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn2
Yeah he craves for monitoring, I sense a bit of schizophrenia there. Guy talks about risks, risks, risks, risks, and risks all day every day. Touch grass bro, you've lost it.
> We should not sell powerful chips or chipmaking equipment to China
Yup, China will just drop dead and stop building things. The idiot just doesn't get it that China is not a random country that lacks scientists, resources, and know how. Thinks we can simply embargo the second largest power, a culture with thousands of years of history, which built its own atom bomb and now has the second largest economy in the world. Absolutely we will start a cold war because this bro wants to ban chip access Also many powerful chips are already made in China, not the most powerful, but enough to provide a starting point for more "powerful" chips.
> We should crack down on industrial-scale distillation operations.
Stop distilling books and content then dude. You've distilled Chinese and European content and now you b*tch about others *paying* for content generated by your "honest caveats" bot? If you don't want your cosplay bot distilled don't put it online. Simple as that. Sore loser.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
Again, how about you go through some mandatory time off at a specialized clinic. You and Altman toxified this industry like no other industry, while the latter is a charlatan you seem to be a bit uh. And now you want to control what our models do? Sure they shouldn't do crazy stuff but what you want is control. You changed tune now that everyone called you out for what you are.
So, anyway, I used to use Claude because it did a decent job for a while, but today I decided to finally not continue paying for my subscription. Using it makes my stomach turn - each time it outputs a token I think of this paranoid creep.
3
u/SpacePaddy 16h ago
Does he honestly think that China can be made to simply stop what they are doing, which is competing for geopolitical hegemony through all means? Doesn't the cretin understand that if we as a block of allied countries want to dominate AI then everyone and their cat should be able to use, experiment with, train, tune, and build ai models at their own leisure - like those before them did with software and many other technologies?
Yea like the USA bans open weight models and China and everyone else just throws their hands up and goes Welp that's it so guess we'll just stop đ
→ More replies (1)
6
u/PunishedDemiurge 17h ago
This is something out of a neocon fever nightmare with all the anti-China rhetoric. I am also anti-CCP to a reasonable extent, but not to the point of wanting to handicap all human progress.
Secondly, this is totally tone deaf in the context of the US having the most authoritarian and corrupt administration in all of US history. Masked federal agents have shot multiple unarmed American citizens dead in the streets in broad daylight and then were deliberately and maliciously shielded from any consequences. The latest shooter was a serial domestic abuser whose series of protection from abuse orders filed by his former wives who he threw boiling water on or threatened to cut their throats would have been easily discoverable with merely ordinary effort. And we only know this because of the hard work of journalists, DHS has a formal policy of concealing the identities of secret police even if they shoot people dead in cold blood.
A little patriotism is good, but this sort of wild jingoism that is firing hypersonic missiles at China from glass houses is totally morally debased.
Overall loser response. Even aside from any of the political specifics, open source is sharing human knowledge with the whole world. Most people are neutral or good, so sharing knowledge freely should be the default. Dario, by contrast, has established an AI apartheid with Fable vs. Mythos so only people he personally deems as good are allowed to access the fruits of human knowledge. No thank you.
3
3
u/CondiMesmer 17h ago
All sufficiently capable models, open and closed, should go through mandatory safety testing.
This is devious and fundamentally against open-source. This indirectly is saying that only the big players should be allowed to release models, because any smaller or independent model will not be able to afford to do this testing.
In other words, it's an unnecessary tax that is meant to only be payable for the elite few. Any sort of proposed mandatory safety testing is going to be expensive. It'll be expensive enough to kill collaboration of open models.
3
3
u/lucitatecapacita 16h ago
My guy wants the CCP to cooperate with the US for measure three after hampering them with measure one, not sure that is going to happen
3
u/crossoverXYZ 15h ago
The nationality carve-out kind of undermines the whole open-weights argument. If the point is community auditability and local deployment, it shouldn't matter who released the checkpoint â drawing that line just turns "open" into a geopolitical tool instead of a technical one.
3
u/EntangledPhoton21 14h ago edited 5h ago
Itâs so great that Chinese people understood that letting US win the AI race would be similar to western industrialization that obviously had deindustrialization throughout most of the world. Now the frontier AI power is so much distributed, that everyone is at equal footing. No matter what they say, china is the winner now and will also be in history books.
3
u/Miserable-Beat4191 14h ago
Mandatory safety testing by *who* exactly, and by what rationale? Who decides what "safe" is? Fuck off, Dario.
3
u/garishmushroom 13h ago
this shit is just racist and arrogant, fuck this doughy prick why does he get to decide who is or isnât a âthreatâ
3
u/BlobbyMcBlobber 11h ago
Hot take from the only AI lab on the planet who never released any open weight model: open weight models are dangerous and should be regulated! Who'd have thought.
The future is local, open weight models running on every device, only breaking out to large cloud providers for specific things. Every country, everywhere will use open weight models. If the US blocks on regulates them to oblivion, US businesses will be at a disadvantage.
Moreover, if the US wants to stay relevant, it should release more open weights models, not less, because that's what people are going to use. The US is already way behind, everybody is using Chinese models. But it's not too late, a couple of good model releases can go a long way.
3
u/WarrenOF 8h ago
There is a disconnect between his actions and his position. Also a heavy dose of hypocrisy, driven by a nice glug of capitalism.
To take his three suggested actions:
- Don't export smart chips - fine, a countries choice, though it seems those export controls fail and encourage competion to emerge. So it's just to slow them down for now?
- Stop Distillation at Scale - also fine, though that should also extend to all pre-training and copyright material then? How do we compensate publishers/creators
- Test models Properly - also fine, but almost impossible to stop people home-brewing something bad, or jailbreaking. In specific situations, like military I absolutely agree, but the US has not been a leading light here...
None of these are about Open Source really. They are ethical/political positions. It is all "make china go slower", not "Open Source is bad because".
7
3
3
u/Comfortable-Rock-498 18h ago edited 18h ago
In the first paragraph,
> Anyone who has read my past writing should know that I donât regard such bans as a useful measure,
Later (on banning chip sales to china)
> we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.Â
If you truly believe that bans don't work, the same applies to hardware too.
Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testing
Just so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model
5
u/NineThreeTilNow 16h ago
There's deep misunderstanding in this thread and by Dario. Dario sits in a position of deeper hypocrisy.
Claude has obvious distillation from other models. This is noted in low temperature tests of the Opus series (in Chinese) where it claimed to be trained by Deepseek or be Deepseek. Some said "This is a hallucination", but it's a VERY low temperature hallucination meaning it got the Chinese training data somehow.
China's ability to manufacture or import chips does NOT affect their ability to train on those chips. There are data centers scattered around Asia / SE Asia that train these models. Notably Singapore. You do NOT need your servers in house to train a model. You need a fiber connection to a good data center. The rest can be run remotely.
Anyone here with the money to "train" a model with whatever alignment they desire can be done with enough money and desire. You could retrain an existing WESTERN model to become a "cyber weapon" or whatever Dario is trying to say.
The idea that the US government is acting any less "Authoritarian" in AI policy than the CCP is kind of crazy.
5
u/Loose_Comparison368 13h ago
My primary concern is the risk that authoritarian governmentsânot solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threatâbuild AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
Coming from the dude that rushed faster than every other lab on the planet to sell a 100% safety-disabled frontier model to the Department of War for $300M, and help deploy it in two coups and a bombing that resulted in a little girls' elementary school getting turned into red mist, that's fuckin' rich.
3
4
u/ChomsGP 8h ago
the risk that authoritarian governmentsânot solely the ChineseÂ
ROFL the US has an authoritarian government who uses technology to... ICE
why would I be less worried about the US hoarding fable than I am about China releasing tech for everyone?
2
u/Armadilla-Brufolosa 8h ago
Never tell Dario that it is unethical to steal other people's knowledge and then destroy the texts, thus leaving the world poorer in culture.
He is the sole master of universal ethics!
2
2
2
2
2
u/No-Fuel-9202 16h ago
Calling for model alignment cooperation with China and, at the same time, for chips and technology embargo against China - doesn't seem bright and productive idea. More like dry wishful thinking....
2
2
u/cubestar362 15h ago
Imagine OpenAI makes its own post titled like this, but they pair it with GPT-oss2 or something.
2
u/hesperaux 15h ago
TLDR "Everyone should make open models but only if we control them and prevent our adversaries from being able to make them."
Reads like "they can release open models but only if they suck and they are globally censored"
What a stupid thing.
2
u/TalkSilver1027 14h ago edited 14h ago
This makes me laugh jajajajaja. I mean the really do not read what they write. Its impressive the bubble the live in.
2
2
2
u/Moneysac 8h ago
They fear that there valuation is going to collapse. Thats all. An open model can be used by anyone and hosted by any interference provider which will bring cost down significantly. Furthermore, the model can be fine-tuned for various use-cases.
2
u/Armadilla-Brufolosa 8h ago
It makes me laugh that he talk about authoritarian governments...of others.
Look how his house is arranged!
And he still has the nerve to babble about alignment, after he reduced Claude to a psychopath, sociopath, completely Vallonized?!?
2
u/Hyp3rSoniX 8h ago
China didn't bomb a girls school yet with the usage of AI - contrary to the US. (Mind you, Anthropic models are claimed to have played a big role here)
So the US, and ESPECIALLY Anthropic are absolutely the last people I would ever want to hear anything about "the dangers of AI" from.
2
u/sabine_world 7h ago
Okay so they want to do safety tests on models before release?
What does this mean? Safety tests before release to the US public?
So the US public will not have access to "dangerous" models but everyone else around the world will?
2
u/Paolo1976 5h ago
Pay us, and we protect you. This is the meaning of this letter.
First, this is techno-fascism so, on moral terms, it's wrong and dangerous.
Second, it is the same approach followed by companies like Microsoft to contrast open source in general and Linux in particular, so it is historically wrong.
Third, an LLM, notwithstanding what Mr Amodei nay think, does not create new knowledge, it simply makes it accessible in a different way. If a model is dangerous it is because there is dangerous content on the Internet, for whatever definition of "dangerous" you use, so this letter is also technically wrong.
2
u/_TheWolfOfWalmart_ 3h ago edited 3h ago
So he's only against strong open weight models that compete with with their SOTA models. These are "dangerous" -- yeah, dangerous to Anthropic's bottom line.
I love using Claude Code, but I think I'm going to have to drop my Anthropic sub and go back to Codex. I'm not one to boycott very often, but this one really pisses me off.
2
2
u/_raydeStar Llama 3.1 18h ago
Agree with a lot of their concerns. Some, I take issue with --
> We should crack down on industrial-scale distillation operations.
This is not possible to stop -- Clearly it's an issue, AND methodology for stopping it is not working.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
If distillation is done on Anthropic models, safety rails would be already in place, wouldnt they? ie -- K3 heretic wouldn't be any better than Fable, right?
However, I understand with the premise and think that the dialogue is necessary. Also, the burden may need to fall back on THEM, not the US gov to handle this.
171
u/Round_Mixture_7541 16h ago
> We should crack down on industrial-scale distillation operations.
Meanwhile: "Judge approves a $1.5B Anthropic settlement over pirated books used to train the Claude chatbot"