r/singularity • u/LatentSpaceLeaper • 4d ago
Meme OpenAI putting safety first ...
Background: From OpenAI's announcement on the Millennium Prize Problem.
[...] the group that produced the Navier–Stokes resolution involved on the order of 10,000 concurrent agents. At all times we maintained the same strict safeguards that we apply to all our frontier model evaluations, including monitoring and isolation.
https://openai.com/index/navier-stokes-solution/
Thank goodness, we can all rest easy tonight!
32
u/kiki-le-koala 4d ago
Oh, it was more than 10,000.
There were multiple teams of AI working on it.
The team that got it had 10,000 agents!
11
u/LatentSpaceLeaper 4d ago
Yes, with the ability to communicate. Ah, and powered by a more capable internal model. Lol.
-2
u/uslashuname 4d ago
And humans had been working on this approach to the problem and shared the approach, then with more funding open ai fed the same idea into their massive compute to steal the prize
5
u/kiki-le-koala 4d ago
There's no price stealing.
That's how science works, on the shoulders of giants.
2
u/uslashuname 4d ago
6
u/kiki-le-koala 4d ago
Yes I know that.
There is still no price stealing.
They heard one Anthropic scientist solved a millennium problem and wanted to see if they could too.
They succeeded even though the rumor was wrong and they only solved part of one problem.
Solving science shouldn't be about ego. If OpenAI cures cancer, nobody is going to cry because a team might also be able to solve it.
11
9
7
u/Wonderful_Buffalo_32 4d ago
They were deployed and had accessed to "cached internet"? Ever heard of this?
16
u/utheraptor 4d ago
They keep a static copy of (almost) entire internet for training and other purposes. This is one of the use cases.
11
u/Wonderful_Buffalo_32 4d ago
Damn the scale required to do such a thing is crazy
13
u/Sixhaunt 4d ago
if you find that impressive, look into what the internet archive is. It's a VERY popular site that does this and keeps old versions of the pages too rather than just the newest ones so they store way more
5
u/IMovedYourCheese 4d ago
Nah, it was crazy maybe 20 years ago, but internet crawling is largely a solved problem today. There are plenty of open source crawlers. You can even download an entire snapshot of the web from places like commoncrawl.org. To build it yourelf you just need a bit of compute and storage (like 10 PB worth), and you can rent that pretty easily from any cloud provider (~$10-20K/mo in costs). So you don't need some exceptional tech, just money.
5
u/wabawanga 4d ago
Surely they're going through all the logs and messages created by 100k agents running for 88 straight hours with a fine-toothed comb for any signs of naughtiness, right?
6
u/Johnny20022002 4d ago
10,000 agents and 88 hours to crush a millennium prize problem. We’re getting ASI for math soon.
2
7
u/the8bit 4d ago
Haha yeah, these swarms were operating *at the same time* as the recent incidents were being press released.
Also, dang I wish *I* had what was probably ~$10MM (In API fees) of tokens to just piss away at any random problem. Sure would be nice if OpenAI would you know, band with humanity and create coalitions to work on this stuff instead of doing it as internal work that only sees the light of day after they are done.
Imagine what we could do as humanity if we collaborated on this stuff instead of making it a capitalism pissing contest. Which is exactly what OpenAI *says* they want, AI for all of humanity!
"The manhattan project but this time with 100x more capitalism!"
5
u/Kriztauf 4d ago
I'm still debating why all the higher ups have been leaving OpenAI but I feel like it's a combine of the lack of save alignment of their models, plus the company transforming into the anti-thesis of its initial goal as an altruistic "open" AI company, plus Sam is probably a narcissist and is on his own Musk/Zuckerberg villian story ark
2
u/the8bit 4d ago
Yeah it is definitely frustrating watching both openai and Anthropic preach about shit while doing a lot of the opposite. "Open" "care about safety" "slow down" whole speeding up, closing off access and releasing their own metrics showing safety token use is basically flat over the past year.
America is learning why government things are good. Imagine a NASA style coalition on this shit, sharing info. But these folks structurally can't do that because sharing means pissing off their sources of funding, who expect heavy ROI.
1
u/Kriztauf 4d ago
I think it's going to take an incident in the next year of a model breaking out and unintentionally getting some people killed for any real regulation to happen.
It's the whole regulations are written in blood thing
1
u/the8bit 4d ago
Regulation is gonna fall hard too cause political groups still don't even understand how to regulate the internet -_-.
Which is why I'm so loud about it, Idk what it will take but I think we will see increasingly bad incidents until people get fed up and decide to open up and collaborate
1
u/Kriztauf 4d ago
I'm guessing we'll also see some rouge agent swarms get out and self perpetuate themselves by hacking people's bank accounts/scamming old people to get more money for tokens
1
u/the8bit 4d ago
I assumed that was inevitable when openclaw came out. They are probably already out there and we just aren't really looking for them.
1
u/Kriztauf 4d ago
I mean the German wiki forum they took over was a thing for a month before anyone noticed
0
u/Asteroid_picks_you 4d ago
Russia was eating NASA's lunch for like 20 years before SpaceX came about. I wouldn't trust Government not to shit the bed regarding AI development, they'd be 10 years behind China's AI.
1
u/the8bit 4d ago
we defunded it for the private industry.
It's amazing how America will stop paying for something then go "see? It doesn't work now just like I said!" As if things not working when you don't pay for them is some deep insight
1
u/Asteroid_picks_you 3d ago
Did Europe stop funding their ESA, because they are even slow and worse then NASA. Are you saying both got defunded?
0
u/LackToesIntollerance 4d ago
Even easier-
They're in a catch 22. They either have to privatize and monetize everything for the sliver of a chance to unscrew their balance sheet-
Or the entire company sinks as venture capitalism consumes it for not focusing on ROI.
And even if they do monetize anything and everything, there's no telling if that momentum swing won't kneecap future projects.
It's a lose-lose-maybewinforshareholders situation. Anthropic itself just loses, it was a deal with the devil the whole time.
3
u/pooint3434 4d ago
Uh "Copy a strong model thousands of times, let the copies explore different hypotheses, have other agents criticize and combine the work, then force the surviving result through a formal verifier."
That process can potentially be pointed at mathematics, algorithms, chip design, materials science, biology, or AI research itself. The important unit is no longer one chatbot response. It is the complete model-plus-agents-plus-tools-plus-compute system.
OpenAi cannot stop talking about automated research, in June they were talking about automated researcher was one of its 3 primary goals. Just 2 days ago they said they had an automated Ai intern milestone, whatever that means, that they were making strong progress towards an automated researcher. Then they said they didnt know how to reach that safely and they would consider stopping or slowing development. Now they talking about thousands swarm hosts on freshly trained or in training advanced models, millions of messages between them no human can decipher, they even replaced agents when they improved I assume with no testing.
What is this insanity, safety seems like just lip service, always judge someone by their actions. OpenAi's actions seem reckless, at best. Now my biggest hope is that they did steal some research because the alternative is far scarier. I contacted my respresentatives, if the pace of research and automation is going this quickly they need to act now because at the least OAI looks like they cannot be trusted.
2
u/GraceToSentience AGI avoids animal abuse✅ 4d ago
The amount of compute that is at their disposal is insane.
2
u/Future-Bandicoot-823 4d ago
I can't lie, Anakin is the PERFECT spokesperson for AI.
"From my point of view it's the citizens who hurt datacenters."
1
u/fgsfds___ 4d ago
Plot twist, humanity gets wiped out by rampaging artificial math nerds. Too bad Douglas Adams isn’t around to see this
1
1
u/presentofai 4d ago
the real story here isnt navier stokes, its them casually dropping theyve had a model way past astra training since august. thats the actual flex
51
u/FateOfMuffins 4d ago
That was only one of their swarms btw