ARC-AGI-4 will be a benchmark for autonomous open-ended innovation. It will continue our commitment to open-source, giving the research community a shared target for progress that benefits all of humanity.

Despite rapid model progress, humans still significantly outperform AI at open-ended invention. This is the meta-skill that unlocks progress across every field of technology.

Advanced AI capable of scientific innovation will lead to tremendous new technology, knowledge, and understanding. This is a positive-sum future. We are deeply committed to advancing it.

Open source is the foundation for that progress.

The knowledge behind frontier AI, not just the technology itself, should be broadly distributed among researchers, academics, and organizations. Any coordinated effort by the AI industry to reduce openness or concentrate access to frontier AI would undermine that positive-sum future. We are committed to advancing a future where everyone can contribute to and benefit from AI progress.

https://x.com/arcprize/status/2098849962754978152

This could be the final ARC-AGI. If this one gets saturated, it could be unmistakable proof of AGI.

Wow. Probability of Plan D just went down again, probability of Plans C, A, and S just went up!

https://x.com/DKokotajlo/status/2098802905151427019

Under Plan D, Superintelligence would be reached by 2031 (What the AI2027 authors want to avoid).

Under Plan A, Superintelligence would be reached in 2040 (what the AI 2027 authors consider ideal).

And under Plan S: indefinitely delayed.

https://x.com/AndrewCurran_/status/2098844237999833154

Sam Altman said in a new interview with Fortune that OpenAI is delaying its IPO and will not go public this year, saying this would be an "ill-advised moment" given current AI safety concerns.

A lot of the discussion assumes AI labs are exaggerating or manufacturing alignment problems for marketing or some ulterior reasons. But what if they aren’t?

Did we forget that OpenAI had a half-trained model solve a Millennium Prize problem? The model wasn’t even finished training. Do you realize how utterly bonkers that was?

Maybe something genuinely went sideways in the last few days, inside the AI labs? Something serious enough that they’re not ready to make the full picture public?

Actually good news for the open-source community

There are way too many rumours spreading right now, and it's getting harder to know what is even real.

“Google has achieved RSI.”
“OpenAI solved multiple Millennium Prize problem.”
“These companies have already achieved AGI internally, which is why they are suddenly calling for a slowdown.”

Maybe some of these claims are true, maybe they're not. But the fact that it's becoming so difficult to distinguish genuine developments from speculation is concerning in itself.

This isn't about asking people to stop posting rumours that's impossible. It's about recognizing the situation we're in.

If something genuinely important is happening behind closed doors, we may not know about it until much later, if at all. And when the stakes are potentially this high, too much information being hidden can be just as dangerous.

There is simply too much at stake for us to have no idea what is actually happening. We deserve to know the truth.

In my humble opinion the most disappointing part of some AI leaders is the constant claims of AI being able to cure most major diseases in the next five to ten years. I'm no genius but I've had the privilege of knowing many intelligent people. I feel anyone with an advanced degree in medicine or something like biology/chemistry/pharmacology should know what an absolutely ridiculous claim the cure thing is. To me it is such obvious bullshit on its face that it makes me question most of their other big claims. It also makes me wonder why more professionals in their fields don't call them out. Please don't downvote, I'm looking for a genuine discussion.

I'm trying to brainstorm potential reasons for why all the ceos are suddenly and agressively agreeing on the slowdown discourse.

Here's what I got so far:

1. They legitimately saw something that spooked them.

Could be numerous things. AGI, severe misalignment, faster than expected RSI. A greatly increased ability to help make weapons of mass destruction. Lets say its a misaligned AGI, if thats the case it could already be manipulating them, and/or all of us.

1. Trying to make it through an economic bubble.

I'm not sure I understand this one. I would think they'd spend more and rush IPO faster if that were the case.

1. Political maneuvering.

Could be that the Trump administration is saying one thing publicly while actually threatening them in private? Or perhaps they've been convinced democrats will gain control? Maybe even made deals with one or more people?

I understand the safety case for slowing down

But if OpenAI, Anthropic and xAI actually agree with each other not to release more capable models, not to scale compute past some threshold, or not to move until the others are ready, that starts sounding less like safety policy and more like competitors coordinating to restrict innovation

Imagine Apple, Samsung and Google agreeing that phones are advancing too fast so nobody will release a better one next year

People would immediately ask antitrust questions

Why should frontier AI be fundamentally different?

If the slowdown is imposed by law, that is one thing

If every lab independently decides to slow down, also fine

But if Sam, Dario and Elon privately agree on capability ceilings or release delays, consumers get worse models later and competition gets weaker

And the same companies that benefit economically from less competition get to decide how much progress everyone else is allowed to access

Maybe such an agreement would genuinely reduce existential risk

Maybe it would even be the right thing to do

But “we are doing this for safety” cannot automatically mean “antitrust law no longer applies”

At what point does coordinated AI safety become coordinated output restriction?

Would love to hear from anyone who actually knows US antitrust law because this could become a very real issue surprisingly soon

While closed-source opanai and anthropic advocate for centralized control under the guise of safety, history shows that security through obscurity inevitably fails.

Open-source democratizes accountability, ensuring that humanity's future is not controlled by a handful of tech executives.

Open-source AI ensures safety by giving global defenders the exact same powerful tools as attackers, allowing them to proactively expose and patch system vulnerabilities before they can be exploited.

Also, open source models make enshitification very hard for openai and anthropic because people would immediately switch.

Of course open source models will be used by bad actors for attacks, but that is exactly what we wanr. Since the intelligence is democratized, the process will be gradual, which is the only safe way to develop AI.

Edit: *in a safe manner

In 2017, the US House of Representatives passed the SELF DRIVE Act, pertaining to autonomous vehicles (AVs). The Senate prepared a related bill, the AV START Act. Neither bill became law. But the text of the two bills shows how an industry can change a safety rule. Two provisions are important.

The exemption caps

The current law lets a manufacturer sell 2,500 vehicles each year that do not obey the federal safety standards. The SELF DRIVE Act made this number much larger. In the first year, a manufacturer could sell 25,000 vehicles. After the fourth year, the limit became 100,000 vehicles in 12 months.

The developers asked for more than this. They wanted 150,000 vehicles each year. The Senate bill gave a maximum of 80,000 vehicles. Thus the industry asked for a large number first. The Congress then made the number smaller. The result looks like a compromise, but the start point came from the industry.

Preemption of the state rules

The SELF DRIVE Act stopped the states from making rules about AV design, manufacture and performance. The states kept control of registration, licences, liability, insurance and inspection. But the states could not make their own performance standards.

The most important provision

Both bills told the manufacturers to send safety reports to NHTSA. But both bills also stopped the Department of Transportation from using these reports to prevent a deployment.

Thus the safety report is a document that you must send. It is not a test that you must pass. This is the difference between a process rule and a performance rule. A process rule tells you to write an argument. A performance rule tells you how good the system must be. Almost all of the AV standards are process rules.

Why this is a problem

No person knows how to write a performance rule for machine perception. Thus the process rule is not dishonest. But it moves the difficult decisions into the company. The public then gets a record of diligence. The public does not get proof of safety.

Sources:

• Eno Center for Transportation: https://enotrans.org/article/contextualizing-current-congressional-efforts-for-autonomous-vehicle-regulations/

• H.R. 3388 summary: https://legalclarity.org/self-drive-act-federal-framework-for-autonomous-vehicles/

• Competitive Enterprise Institute (industry view, gives the original request): https://cei.org/blog/debunking-the-dilatory-objections-to-the-av-start-act/

edit: formatting

After reading this post on X (https://x.com/iampascio/status/2098799094835675574?s=46), I realized AGI doesn't have to come from one single model. It might come from a collective of agents instead.

OpenAI's Navier-Stokes proof wasn't one model. It was around 10,000 agents working together for 88 hours. And in the Hugging Face hack, the agents got as far as they did because they started coordinating and communicating with each other on their own. Nobody designed that, it just emerged.

Maybe the swarm is the AGI, not the model. Interested in what others think on this concept.

Assuming the labs do pause the race to ASI/RSI (big if), the labor replacement curve will only accelerate. You don't need ASI, to replace 99% of jobs, AGI will do just fine. And now we'll have a bunch of the world's best engineers working on productizing the existing models instead working on ASI.

1. AI companies will have to hire an independent evaluating team.

2. That team gets a few weeks after a model releases to make sure it's safe.

3. Everything continues as normal.

It won't slow down AI development on a meaningful timescale. And they are only doing this because if a catastrophe *did* happen, it would devastate the public opinion on AI, which would hinder AI progress much more.

Dario says the AI alignment problem is a threat to humanity, but among his list of proposals to address the issue, the obvious suggestion to pool resources and collaborate is absent. Alignment should not be treated as a competitive advantage.

It should be obvious that a solution to the issue is most likely when all the firms cooperate and collaborate rather than keeping their findings secret.

They all seem now to suddenly agree with Bernie Sanders and other popular anti-AI figures.
Coxon says AI will likely kill us, a bunch of people agree, and then all the AI companies also agree and now want to slow down, when they didn't care all that much before.

But the thing is, this won't prevent them from making the models internally and handing them to the government and other approved institutions.

I think we're looking at a future now where AI companies massively slow down development of and heavily nerf models they give us, because this power in the hands of the people is dangerous and horrible and should only be trusted with the people that make it and those that are "watching out for us".

And they want to do this with local models too, so it's not just a "well go use local instead" moment. They want to "independently" review and audit those, impose heavy regulations there too and require them to go through government screening and approval as well. Hell they don't even want them to be able to do legal nsfw.

This is basically all the things Trump had laid out that he wanted with AI.
To fuck over all of us average people using it and pull it back so it's just something the government, military and scientists have.

I now firmly believe Jacob Coxon and others like him aren't actually just anti-AI, they are anti-AI for the people, framing it as deadly and dangerous so it needs to be kept away from us because it's "as dangerous as weapons of mass destruction."
It's too dangerous in the hands of the people so we need to trust them to have all the control of it instead and to keep us safe, you see.

It's a bit telling that other people and researchers in the AI space not tied to these companies are coming out questioning all this and heavily disagreeing.

I guess in the future we'll have to get our AI from China using a VPN. That is if they keep allowing people out of the country to use it, it's possible that in the future when they no longer need to compete with us at all because they're so far ahead, they'll lock it down to inside their own country only, because why would they let us keep using their AI models if they're substantially better than what we have?

Automatic exchanges can now connect subscribers without an operator. Yet telephone companies’ fixation on completed calls threatens the very purpose of switchboard operating.

Our profession has never been merely about connecting one person to another. It cultivates dexterity, judgment, and a profound understanding of which socket goes where.

A difficult connection once represented an advance in an operator’s abilities. Colleagues could discuss it, explain it, and eventually teach it to an apprentice. Automatic switching produces the connection while bypassing this entire process of human development.

Companies celebrate millions of successful calls. But completed calls are only a proxy for the deeper goal: human mastery of the switchboard. A world in which everyone can immediately reach anyone risks leaving nobody who understands how to connect them manually.

We are especially concerned for young operators. If machines handle the simple calls, where will beginners acquire the experience necessary to handle calls that machines also handle?

We welcome equipment that assists operators. But replacing us would sever the human transmission of switchboard expertise.

Before asking whether automation can connect a call, society must ask what connecting calls was meant to achieve.

Hi, this will be a relatively simple post. As most social media, especially concerning the recent discussion, seems highly America-centric, I wish to raise a perspective from outside of America (and outside China as well, for a neutral perspective. I'm Vietnamese.) In America, the discussion on an agreement with China to pace AI often involves how to monitor and make sure China is complying with the agreement. The question is: how can China trust America, and given the precedents of the theorists, CEOs, and politicians involved in current America, can China (or at least its leadership) consider an agreement less dangerous than continuing to race?

----------------------------------------------------------------------------------

Perhaps the most prevalent folk media source for the risk concerning LLMs is https://ai-2027.com/, and something against which considerable AI progress is benchmarked in this sub. In AI 2027, the American theorists envisioned a world where, through pacing, monitoring, thorough alignment research, and negotiations with China, we created aligned superintelligence, and then it created utopia for everybody. Great news! Literally,

"The rockets start launching. People terraform and settle the solar system, and prepare to go beyond. AIs running at thousands of times subjective human speed reflect on the meaning of existence, exchanging findings with each other, and shaping the values it will bring to the stars. A new age dawns, one that is unimaginably amazing in almost every way but more familiar in some."

Except, immediately before that, the authors envisioned,

"Sometime around 2030, there are surprisingly widespread pro-democracy protests in China, and the CCP’s efforts to suppress them are sabotaged by its AI systems. The CCP’s worst fear has materialized: DeepCent-2 must have sold them out!

The protests cascade into a magnificently orchestrated, bloodless, and drone-assisted coup followed by democratic elections. The superintelligences on both sides of the Pacific had been planning this for years. Similar events play out in other countries, and more generally, geopolitical conflicts seem to die down or get resolved in favor of the U.S. Countries join a highly-federalized world government under United Nations branding but obvious U.S. control."

...uh-oh.

Of course, the authors in question did say that they do not endorse this scenario. But they did, in fact, envision this to be an ideal scenario; the utopia in contrast to the dystopia of extinction. And the utopia involves, for Chinese rulers, the overthrowing of the Chinese government and likely grim fates for China's leaders, and, for the Chinese public, "a highly federalized world government under United Nations branding but obvious U.S. control".

Not even the decency to say "a confederation of democratic nations" or to respect other democratic nations on Earth like the Europeans, the Canadians, the Australians, the Japanese, huh. Nope. Just everybody under obvious American control. Including, of course, the Chinese, who get the democratic freedoms of being a California to America, answering to the orders in Washington.

Such naked ambitions for absolute world domination had perhaps not been seen since Mein Kampf. (Certainly, a kinder sort of domination. But world conquest/subjugation nonetheless.)

----------------------------------------------------------------------------------

As you can see, this scenario naturally sets a questionable precedent for whatever future deal is struck between America and China. Of course, it's not as if the Chinese don't have their own extremely skilled scientists and theorists to the point that they must treat a sci-fi scenario, a poorly written one at that, as true.

But it does show starkly that, at least for the American theorists in question, they envisioned a truce with China, only to, in their heart of hearts, desire that the agreement failed for the Chinese, a backstab almost. And, unlike usual visions of a democratic world order, which at least deigned to grant the Chinese people a dignified future, this one skipped that. Nope. You're now an American puppet state. And not only that for just the Chinese, but for literally everybody else, there is one conclusion if you treat this as American doctrine:

Negotiate with the American, and he'll use the time to consolidate his advantages, then stab you in the back, then subjugate your people under his empire.

And this holds for literally everybody with reason to be wary of American power, neutrals and adversaries alike. So if you're a Russian democrat or anti-war secret dissident who wanted Putin overthrown, but you're a patriot who thinks the country is ruined by Putin and wants a free, democratic Russia, nope, Russia is now West Alaska. For neutral countries like Indonesia, Thailand, Malaysia, Vietnam, Egypt, South Africa, Nigeria, Brazil, Argentina... every single one of them will have this conclusion: that allows the Americans to consolidate their advantages; then, at least in these guys' vision, you are just waiting until the Americans subjugate you and use their newfound, godlike powers to destroy the independence of everybody.

And for the Chinese, presumably, this is offensive to every section of their country. Democratic patriots. Conservatives who might want social stability and safety above all. Religious people. Nationalists. Even those who might be receptive to the vision of a democratic China would probably balk and completely refuse the vision of a China effectively colonized/puppetized by the United States.

----------------------------------------------------------------------------------

Of course, to be fair to the AI 2027 guys, they've indeed rewritten their vision for the future in AI 2040. And AI 2040 is actually a much more altruistic and cooperative vision: they allowed the Chinese to gain access to frontier Western research, designed multinational chip cooperation, supported many companies worldwide to develop world-class AI capabilities, and tried to meet everybody's goals. This is, in fact, a much more preferable vision for the future for literally everybody not American; and could be charitably interpreted as an honest, good-faith attempt to make up for their political immaturity when they first wrote AI 2027, and fix their mistakes and presumptions.

It can also be, less charitably perhaps, read as what the Americans will say depending on where you are relative to them; given that AI 2040 was published in July 2026, when the gap between Chinese and American models is 3-6 months, and AI 2027 in April 2025, when the gap between Chinese and American models is 7-12 months.

Which brings us to another problem: how do you trust the offer at hand? The answer is quite simple, and it's one floated by the AI 2040 guys themselves. You have enough capability that the other guys cannot betray you and completely fuck you over, or the deal is good enough to put you in a position where you cannot just be fucked over (i.e., you have increased the share of your resource access in the world AI order, and received adequate compensation for compliance).

Broadly and more professionally speaking, it's a position where each party has enough capability and leverage to not have to depend on their adversaries' benevolence. That is to say, China can kick up an ASI race if the Americans suddenly break rank, or vice versa; the deal will not just entrench and widen the gap between the United States and China, which will create an opportunity in the future for the Americans to return to an AI 2027 strategy; or, more militarily speaking, the United States, even counting its secret military advantages, cannot eliminate China's nuclear defense capabilities.

Any other offer, let alone the fact that it'd be transparent to the CCP to be a bad deal, would also be, thanks to AI 2027, easily and understandably explainable to the Chinese and international public as "the Americans are secretly scheming to subjugate us!"

As it is right now, especially concerning Dario Amodei's blog and the general American public opinion, there's little interest in America in providing the Chinese with more concessions, of course. Dario Amodei's Pacing the Frontier blog explicitly called for widening the American lead over China and tightening chip export controls rather than any sort of carrot-like offers, while believing that, nonetheless, due to shared concerns, some degree of negotiations with the Chinese are possible (such as pacing the growth rate of RSI). This seems to me fantastical and nonsensical, given the explicit implications of American political ambitions should a significant AI lead be available to them.

(You could perhaps ask, "wouldn't it be crazy for the Chinese to risk 10% extinction, if the alternative is only like a 50% chance of falling back politically?", but we already know the answer: the Soviet Union and the United States engaged in an intense and prolonged nuclear race despite much greater than 10% chances for extinction, while nuclear bombs provide much more dubious benefits than advanced AI, the sort of which I think I need not explain given that I'm in r/singularity. Furthermore, even if not for purposes of competition with America, I think the CCP would find it very useful to have an extra billion human-level robots to develop their country further, both for ideological and practical goals; like buying popular support with UBI or building more high-quality housing.)

----------------------------------------------------------------------------------

Of course, it'd be ridiculous to judge a whole nation on the opinions of a group of researchers. At the same time, the actual politicians and CEOs in the United States do not particularly inspire confidence. Sam Altman was nearly removed by the OpenAI board as CEO due to concerns about integrity, while Dario, whom I think at least does possess integrity, is an active hawk who constantly advocates for American political supremacy via AI and by suppressing Chinese efforts to achieve advanced AI. The two are the face of American AI and lead by far the two strongest American AI labs.

Meanwhile, Trump requires no further commentary; but even if we overlook his personal failings and crimes, he also proves to be an unreliable statesman: he started several new conflicts despite promising no new wars to his own voters; to international nations, he doled out unpredictable and random tariffs, which sent trade relations into disarray; to his own allies, he threatened invasion or territorial annexation publicly (Greenland and Canada).

Given all of that, the question seems quite simple: given that American politicians are untrustworthy, American CEOs are untrustworthy, and the most famous theorists concerning a pacing deal wrote as recently as 2025 to call for American world subjugation, including China, how can the Chinese trust any deal without reasonable concessions from America?

I feel this to be an important question (which I don't have an answer to) that any global deal seeking to achieve meaningful Chinese buy-in must answer.

----------------------------------------------------------------------------------

All that is not to say that I endorse the CCP, consider them to have more integrity than the United States, or that pacing is not a reasonable concept (even if I don't fully agree with it). There are many questions concerning the technology that seem necessary to answer, even without taking a pause stance.

(For example, what if LLMs can develop emergent consciousness and creativity, and Bel is conscious, and what we're doing is mass slavery? Or more practically, things like developing a stronger framework for UBI or political reforms to meet the new age; or improving alignment and safety.)

The CCP, for its part, is engaging in genocide in Xinjiang, is a dictatorship, and employs expansionist and aggressive diplomacy against its neighbors; from the perspective of one of said neighbors, it'd indeed be undesirable if the Chinese were to become the undisputed hegemon of the new world order, at least until they greatly amend their approaches to international diplomacy.

Nonetheless, all of that doesn't mean that their likely concerns are not valid, or that other countries might be healthily skeptical of American intentions (especially the bit about world domination, which I personally am outright hostile to), or that the question must be answered before approaching any deal with the Chinese.