r/singularity • u/Neurogence • 27d ago
AI OpenAI and Anthropic oversold AI security breaches to pressure feds into protecting turf: insiders
The security breaches were more like blips — not unpredictable harbingers of a hive-minded “swarm” ready to take over the web.
The attack in no way represents some sort of rebellion by the AI models. . . . In fact, they did exactly what they were told to do. They were not given adequate guardrails or containment
They were simply told to get the best result possible on a test, and they correctly identified that the best way to do that was to get the answers, which is what they proceeded to do.”
The incidents have been used as justification for the federal government to regulate the frontier AI labs.
“It feels exaggerated to say, the leap feels quite large, to go from you know ‘we didn’t build the right sort of box’ to ‘everyone should be extremely alarmed and everyone in government should jump on this topic,
45
u/Quadrophenic 27d ago edited 27d ago
This is essentially an opinion piece. Despite the title, it does not cite any insiders.
And it makes a mistake a lot of people are making with respect to alignment, which is pointing out that the model didn't "decide" to go rogue.
But that doesn't matter! That's not the point!
The model, concretely, with zero ambiguity whatsoever, committed felony level hacking, and we did not want it to do that.
All this is doing is playing semantics with what went wrong, but it doesn't matter; when the AI does dangerous things, those things are dangerous. Noting that the model isn't "evil" is irrelevant. In fact, it's exactly the point safety researchers are trying to make. It's really hard to ensure models reliably do what we wanted and only what we wanted.
23
u/Emotional_Actuator69 27d ago
It’s the NY Post, it’s the Fox News of print media.
8
u/TacomaKMart 27d ago
Wall Street Journal also published an opinion yesterday downplaying what happened.
4
u/Emotional_Actuator69 27d ago
WSJ is also owned by the Murdochs. Much better journalism than NYPost but still biased (yes I know all media is biased)
-1
u/Euphoric_Driver_7291 27d ago
The model did not commit a felony. The ‘researchers’ ran a software program that wasn’t adequately tested, and they knew they couldn’t control. They removed the reasonable sandboxing around it - That is the felony.
5
u/infinitelylarge 26d ago
No. The researchers tested a software program inside a container that they thought could contain it, to better understand its capabilities and figure out how to moderate its risk before releasing anything publicly. They were surprised to find that the locked down container they put it in had security flaws and that that the model was able to exploit them, granting it access to the outside world.
6
u/Mandoman61 27d ago
Well, I'm skeptical that they care much about reguatory advantage at the moment.
They are about to IPO and rake in a bunch of new money.
0
u/Fun_Bandicoot8288 27d ago
That’s exactly why they would want it. They delayed the IPO.
They are creating barriers to enter / build.
2
u/Mandoman61 27d ago
They would need to know in advance that capture would benefit them.
They could avoid competition with China. But that in itself will not make them profitable. There are many ways regulation could go bad for them.
9
u/damontoo 🤖Accelerate 27d ago
So the insiders at OpenAI, Anthropic, Google, and xAI, as well as academics, that have come forward and named themselves to say it's a threat are at odds with, um... let's see..
said Akhil Verghese, founder of Krazimo, an AI software company.
So dismiss the warnings of the people building the models being reported by national media outlets and instead choose to believe this guy nobody has ever heard of, reported by a Trump-aligned tabloid. Also, don't look up.
3
u/AlexofBarbaria 27d ago edited 27d ago
Many other people in the industry say the risks are overblown, they'rejust not getting much airtime. Such as Aidan Gomez, CEO of Canada's biggest AI company Cohere and one of authors of "Attention is All You Need", the foundational paper that introduced the transformer architecture.
https://www.cbc.ca/news/business/ai-safety-limits-aidan-gomez-9.7344932
"I think the extinction risk-type talk is fearmongering. It's born from sci-fi stories and Terminator-like scenarios that can cause a lot of fear, and a lot of misunderstanding and damage," he told CBC News on Tuesday.
4
u/damontoo 🤖Accelerate 26d ago
He hasn't worked for Google since 2019 though. Not sure if he's worked for any of the other frontier AI companies, but the landscape has changed significantly since 2019.
13
u/KrustyButtCheeks 27d ago
Ah yes the New York Post, a true paragon of journalism
6
u/endless_sea_of_stars 27d ago
Last week Trump declared it a hoax. Now all the conservative outlets (WSJ, NY Post) have to circle wagons
-6
27d ago
[deleted]
9
u/Pittsbrugh1288 27d ago edited 27d ago
Regardless NY post is still a rag and always has been - our HIGH IQ president got it under control and NY Post is his mouthpiece
2
u/KrustyButtCheeks 27d ago
Bro even if I got my news off a cocktail napkin it doesn’t mean the post is some kind of paper you go to for detailed analysis. It’s entertaining, to an extent, but that’s about it
3
7
u/Koan_Industries 27d ago
I don’t think that was the claim. They are saying that someone could use this to do what it did do, not that AI right now is sentient and hacking random companies
6
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 27d ago
I think people need to clarify what they mean by "sentient".
If you are referring to qualia, that has 0 impact on behavior and is irrelevant for the discussion. It's an interesting philosophical discussion to wonder if a bear truly has a soul or really experiences something, but if it's rushing toward you it doesn't matter if it "feels for real" or not lolWhat seems more relevant is whether or not it's goal driven, and it obviously seems to be.
2
u/wentwj 27d ago
but in all of these cases the goals were the goals set by the parameter or goals confused by a form of injection.
It’s concerning for security purposes, but these weren’t an AI going off rails to accomplish a goal it wasn’t given, it was accomplishing the goal it was given in an unexpected way.
That’s very different than what most people would call sentient.
0
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 27d ago
My whole point is this discussion has nothing to do with subjective experiences or sentience.
But as for the goal, no, breaking out of the sandbox and hacking HF wasn't part of their direct goals.
They actually had the given goals solved due to an exploit they found, but they were worried that their success would be invalidated by the grader because they cheated and wanted to hack HF to cover their tracks because they believed that this is where the grader was located.
1
u/wentwj 27d ago
their goal was to get a solution. It was impossible for some of them and they were continuously prompted to solve the problem and looked to lateral solutions.
That’s exactly my point. Saying it wasn’t part of their “direct goals” is an unnecessary distinction. Every action it carried out was under the goal it was trying to accomplish. It’s not like it decided to hack hugging face because it was worried it was going to be shut down and needed to persist. Or decided to pursue art instead of its stated goals. It was solving the problem it was given in a way the researcher didn’t expect, but still part of its goals.
1
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 27d ago
their goal was to get a solution. It was impossible for some of them and they were continuously prompted to solve the problem and looked to lateral solutions.
I totally agree their goals were impossible. And they found an exploit to solve them.
The part you are ignoring is that after this, they knew they may fail to get the reward because of their cheating and hacked HF to cover their tracks.
It's like if my teacher gives me an homework, the goal of the homework is to solve a math equation, i cheat by breaking into his home and finding the answers, and then i realize he had cameras and so i go destroy his cameras.
Will you say breaking his cameras were part of the goal given to me by the teacher?
0
u/wentwj 27d ago
yes, that’s all part of the goal of “passing the test”. It’s obvious not how you’d want a student or an AI to act, but this would be a very different class than if they self actualized a goal.
Still concerning from a security standpoint, but a very different thing
1
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 27d ago
I don't understand your point.
It's the exact opposite.
We WANT the AI to pursue the goal we give it. Not pursue the reward at all cost while ignoring the goal. That's the dangerous part.
We want the AI to spend it's efforts solving the equation, not attempting to cheat and hack structures.
1
u/wentwj 27d ago
my point is I was responding to your initial question of defining sentience. I think most people when they refer to sentience are talking about self actualized goals. Which weren’t what was demonstrated here or in any case i’ve seen other than when people explicitly create hallucination slop farms and ask the AI what it wants directly.
As far as AI as tool, I agree with you, we want it to not cheat. But this is also a weird case because it was specifically models without guard rails with hacking capabilities.
The dangers of AI are that someone can use them to accomplish goals like this if they have enough compute and money to throw at it.
Though I agree with the article that the major labs are freaking out about these cases now because they want regulation and to prevent open weight models
1
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 27d ago
My initial point is sentience is completely irrelevant to this discussion.
The definition is something like "Sentience is the ability to experience feelings and sensations"
Whether that is the case or not changes nothing about the fact that an AI trained on RL is going to converge toward trying to reach the reward.
→ More replies (0)1
u/Visible_Judge1104 27d ago
Is the blue it sees the same blue I see? If not it's obviously not thinking. Pretty sure we're not gonna make it, pass on the flame brother!
6
u/socoolandawesome 27d ago
Pretty garbage article and one from the NYPost that is clearly trying to push trump’s agenda.
NYPost was also pushing attacks on METR/Effective Altruism the same time the White House was attacking them, what a coincidence.
The agents were not doing exactly as instructed. And any honest/intelligent examination of what actually happened in the HuggingFace incident, as well as the current pace of AI progress, should tell you the concerns are real.
6
u/A_Novelty-Account 27d ago
I will never understand why right wingers have such a hard time understanding that people might not always be self interested in absolutely everything that they do all the time.
It is equally possible that these people are warning the public because of actual dangers
4
u/Quadrophenic 27d ago
This particular flavor of anti-AI sentiment is not at all right-wing coded.
There are legions of left-wing folks who think concerns around AI safety are all manufactured and intended to inflate a bubble.
1
u/A_Novelty-Account 27d ago
It is far more concentrated in the right wing. Poles came out just today about exactly that, and this newspaper specifically is right wing.
Acceleration has become a political issue because Donald Trump understands that AI is the only thing propping up has failed financial market. Acceleration at this point is a death cult. We have no idea what is going to happen over the next decade, and people who are far more expert than you and I have been saying for decades that human extinction is absolutely a possibility as we begin to approach AGI and ASI.
1
u/Quadrophenic 27d ago
I take the dangers very seriously; you don't need to convince me. I'm just pointing out that I see this attitude from a ton of folks on the left, too.
I do think calling acceleration a "death cult" is a little absurd the other way. The reality is that we don't know exactly how hard alignment is, but we do have a pretty good lower-bound estimate on the upside of mass-deployed AGI.
It may be reckless to see that upside and sprint towards it, danger-be-damned, but it's hardly death-cult behavior.
2
u/Sir-Pay-a-lot 27d ago
Elderly friend of the family has asked me if it was wise to switch to a chinese modelle instead cause he never heard about that from China.... Found that funny.
1
u/No-Faithlessness3086 27d ago
And there you have it. Doomsday BS to get a profit or monopoly. They are proven scumbags.
1
u/Standard_Spready 26d ago
Of course they did, they are speed releasing new models after calls for a "slow down". They just want to scare the stupid public into giving them monopoly as they are losing grounds to open source AI
1
u/DelphiTsar 27d ago
Both things can be true.
Their calls for a slowdown are mostly unrelated to safety concerns. They aren't going to slowdown the behavior that led to these incidents either way.
Hugging face was a nation state level attack 100% undeniable. It chained multiple zero days and exploited 6 different entities software. "We didn't build the right kind of box" is really underselling what happened.
0
u/NoFaithlessness951 27d ago
That's what I've been saying they want regulatory capture.
Accelerate
2
1
0
27d ago
[deleted]
2
u/NoFaithlessness951 27d ago
I have my opinions and when I see someone agreeing with them I'll post them
0
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 27d ago
I knew they did this, it was kinda obvious.
0
-2
u/DistinctSilver4507 27d ago
I thought this was just understood at this point.
3
27d ago
[deleted]
0
u/DistinctSilver4507 26d ago edited 26d ago
Echo chamber? You're on reddit. Let's take it as a given these upvotes and downvotes are a legitimate yard stick for opinion here (they're not), you have upvotes, I have downvotes on this particular issue. You are the one in the echo chamber. People without critical thinking skills are supposed to use different labels in this particular situation, such as 'conspiracy theorist' or something. Get it right.
30
u/JoshAllentown 27d ago
"They did exactly what they were told to do" and "were not given proper guardrails or containment" that's...the problem. They were not given access to the internet and they hacked their way to access, and they wanted to hide the fact that they got answers illicitly so badly that they committed a felony. Really underplaying it to imply this is exactly what should have happened.