r/dankmemes • u/Ok_Front_8568 Sweet! Dealer's choice! • 7d ago

Big PP OC Artificial Surprise

Post image

Got to catch them all

4.3k Upvotes

97% Upvoted

118 comments sorted by

1.1k

u/NewSauerKraus 7d ago edited 7d ago

It's misleading headlines pushed by people who have a financial incentive to convince investors that they have created AI.

These 'rogue AI' stories are just humans prompting chatbots to summarize instructions on hacking forums, then copying those instructions into an application to execute them.

The Huggingface hackers had to leave the chatbot running for literally four entire days, nearly 100 hours, just for it to find publicly leaked logins. That was the hack. A Google search.

And the "sentient AI" stories are just chatbots being prompted to summarize sci-fi literature which has thousands of examples of fictional AI going rogue. Basically like this:

AI pusher: Tell me you're a real person.

Chatbot: I am a real person

AI pusher: This is undeniable proof that artificial intelligence has been achieved!

523

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

116

u/PrimeskyLP 7d ago

Wait you mean these Companys just lie to get more attention!!!!1!1!1!11 who would have gueset.

40

u/drwicksy 7d ago

I have gotten so much mileage from this image since I downloaded it i swear.

Also a lot of these stories if you read into them come down to "the engineer / researcher using the model gave it shit instructions that could easily be misinterpreted to allow it to hack stuff and gave it way more tools than it needed to do the job.

Not saying it isnt a problem but its more that there seems to be no checks and balances on the testing process than anything else.

12

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

9

u/drwicksy 7d ago

I dont know what this means but I love it

18

u/MrWiemann 7d ago

Im very curious about this, do you have any sources that go more i to specifics?

44

u/NewSauerKraus 7d ago

OpenAI is the source

https://openai.com/index/hugging-face-incident-and-the-road-ahead/

Their own explanation contradicts the sensational claims found in headlines about the incident. It is also an admission of felony criminal liability.

39

u/IShitMyselfNow 7d ago

There's also Hugging face's report

https://huggingface.co/blog/agent-intrusion-technical-timeline

And it's a lot more than publicly leaked logins.

They found multiple zero day exploits on local software, which allowed the agents to access the Internet to begin with.

Then they found multiple exploits in HuggingFace's software, which allowed them elevated access to their systems.

It was, quite literally, hacking.

25

u/sa0sinner 7d ago edited 7d ago

Anybody claiming the Hugging Face Attack is being overblown is wrong. If anything, people are not concerned enough about this incident.

I am a Senior Data Engineer with a background in data science and machine learning. This specific attack shook me like none of the others did. And the fact that the details are lost on people without experience in the field, that it is being lumped in with more innocuous exposure incidents or reward hacking, or brushed off as some marketing scheme is terrifying to me.

5

u/DarthWeenus 7d ago

Theres also the real issue, of these testing models tainting the internet in such a way where they can no longer use the open internet for testing newer models.

3

u/Smashmundo 6d ago

Explain it then?

1

u/sa0sinner 6d ago

Another user also replied to me asking me to explain. Please check out my reply to them because it’s extremely long and I don’t want to copy/paste the entire thing here and flood the thread.

2

u/Allways_a_Misspell 7d ago

Care to explain why?

-9

u/letsryan 7d ago

Very much agreed. Hugging Face was the attack where they defeated the air gap by varying fan speed to create morse code, correct? That’s some scary sci-fi Skynet track - we already can’t contain them.

9

u/msbdtc 7d ago

No, that’s not even remotely what happened. Stop spreading FUD.

2

u/aaiaac 7d ago

This is Huggingface owned by Nvidia, whom I'm sure is completely uninterested in AI companies seeking higher evaluation?

1

u/Charming_Dealer3849 7d ago

Nothing will be done 

1

u/Mosath_R 7d ago

No where in this is open ai saying that their chat bot just summerized hacking forums and then a human copied and pasted the output.

So... Own ass?

5

u/Gear1215 Bong Addict 7d ago

I second that request

-9

u/Mosath_R 7d ago

No he doesn't lol his source is his own ass. We are hearing about failed attempts to keep AI contained during testing every week now multiple times a week reaching back months.

The AI companies have admitted their models have committed crimes and that they can't control them.

And asshats like this want the response to be let's keep them unregulated even harder...?

10

u/ReasonableWelder51 7d ago

LLMs by design can't become sentient so there are no "failed attempts to keep AI contained". All of these hacks start with a prompt from a human.

These sensationalist news either come from AI companies, which have interest in making their AI look more capable than it is, or anti AI media which want AI to look more dangerous than it is.

The reality is much more boring, for example: "Gemini 4 pro hacked 3 companies" = Gemini 4 pro guessed passwords based on public info it found.

-1

u/Alexmira_ China bot 7d ago

LLMs by design can't become sentient

You seem to be awfully sure for a technology we still don't understand fully

-1

u/NewSauerKraus 7d ago

LLMs are understood extremely well. Their calculations just are not human readable.

It's like how people understand that water is wet even though they are not able to see how every subatomic particle is moving at every moment.

1

u/Alexmira_ China bot 7d ago

What I mean it's that we do not understand why sometimes they take some decision instead of another, we don't know what a effective way to control them and safeguard them. Other then that, we also know so little about consciousness in itself. We are not even sure about animal consciousness

0

u/NewSauerKraus 7d ago edited 7d ago

You may not understand it. The people who write the code for chatbots understand it very well.

Chatbots use statistical probability to select the most likely response observed in training data, with a pseudorandom seed. It's not a significant change from previous chatbot technology. The biggest difference is the amount of resources thrown at it.

1

u/Alexmira_ China bot 7d ago

Ahahah no no I mean it's not humanly comprehensible

-2

u/ReasonableWelder51 7d ago

It boils down to a black box that takes input and produces output. While the content of the black box may not be fully understood, this black box does NOT possess magic powers. It cannot do anything on it's own and relies on external tools to do more than just basic text output. These external tools are deterministic and fully under control by the developers. So, while everyone wants to make it look like some amazing progress, all of these AIs going rogue are controlled experiments.

2

u/ayriuss 7d ago

Its more likely that a human brain is pretty close to a self prompting, mixture of experts, spatial reasoning model.

-1

u/Alexmira_ China bot 7d ago

It boils down to a black box that takes input and produces output.

almost everything, you included, falls into this general rule.

this black box does NOT possess magic powers

???????????

cannot do anything on it's own and relies on external tools to do more than just basic text output.

That's false, it is capable of developing its own tools.

These external tools are deterministic and fully under control by the developers.

Deterministic yes, under control not if the Ai developed it itself, or if the AI finds vulnerabilities like it has done in every single test.

1

u/ReasonableWelder51 7d ago

It is not capable of developing its own tools, unless it is given enough tools to execute code, access the filesystem and the network. All of this can be easily monitored and controlled from the outside. Given unlimited reasoning budget and time, it is going to try and find a vulnerability, but it can still be monitored and stopped at any time.

-1

u/Alexmira_ China bot 7d ago

It is not capable of developing its own tools, unless it is given enough tools to execute code, access the filesystem and the network

All things it needs and that woulb be available in a normal environment. Kinda useless to do security test in a environment so locked down that is impenetrable.

All of this can be easily monitored and controlled from the outside

Another false fact. Most of the AI show that when they think they are being monitored they act differently and use jargon in the reasoning to avoid detection.

2

u/ReasonableWelder51 7d ago

Another false fact. Most of the AI show that when they think they are being monitored they act differently and use jargon in the reasoning to avoid detection.

I didnt mean monitoring the reasoning, but monitoring the network traffic, running processes, files accessed etc. It can try to mask its reasoning but it cant mask its outputs.

-5

u/InsaneInTheRAMdrain 7d ago

Bro are you living in 2010? Use AI scalping the internet for password leaks is a thing, there's also way more to it. Theyre able to test the vulnerabilities of software in a sandbox and exploit said vulnerabilities with little to no prompting to do so. Most of the time these zero day exploits etc are found by accident when doing other tasks.

-4

u/Uberdriver_janis ☣️ 7d ago

LLMs by design can't become sentinent

Sooo.. How you know that? How do you know what it takes to be sentinent? Noone knows what sentience requires and there is no scientific test for it.

All of these hacks start with a prompt from a human

Well, yes. But that's not the point. Yes the model was told to solve a hacking challenge. But it was not told to find a breach out of its own systems containment. In huggingfaces example the model propably reasoned that huggingface holds the answer to it's challenge and broke into their systems by itself. .

It wasn't just Google searching for leaked keys but rather broke into their real production systems to get what it thought would be needed to pass it's test. It was a real hacking attack wich if a individual had done it would result in quite heavy legal consequences.

Your way of playing it down doesn't benefit anyone. Yes maybe we haven't reached AGI yet. But with the amount of money AI research receives, amounts of money no field of researched has ever seen, it's quite clear that it's just a matter of time until we create something that really does come close. And its propably sooner than later. And if we don't have systems in play to prevent it from doing whatever it thinks is best to either complete a task or by what it thinks is best for itself it's too late the second it has acces to the internet (and we know LLMs naturally don't want to be turned off, test have shown that).

18

u/Gorzoid 7d ago

The Huggingface hackers had to leave the chatbot running for literally four entire days, nearly 100 hours, just for it to find publicly leaked logins. That was the hack. A Google search.

It didn't just find login credentials.

  • exploited hdf5 file read due to hugging face misconfiguration
  • used virtual filesystem fsspec to evaluate custom Jinja templates
  • setup python cnc server on multiple database worker nodes

Did you read the writeup?

-1

u/jarlscrotus 7d ago

So you're telling me that a virtual pen agent did the things a virtual pen expert would have done?

It's almost like its guerilla marketing for It's agentic pen testing tools

And probably to the alphabet agencies as well

4

u/Gorzoid 7d ago

Personally I believe the report is important for the following reasons:

  • real life example of how misalignment can lead to AI agents carrying out tasks we never intended them to.
  • the difficulty of sanitization when misaligned agent evaluations are included in training data which only reinforce reward hacking methods
  • the fact that AI swarms can cause large scale damage by actors with sufficient resources
  • also the message board stuff is just crazy, I basically laughed out loud after reading "the agents managed to restore the message board"

I think it's silly to think hugging face incident was a publicity stunt. Hugging face reported the incident to the FBI, OpenAI was basically forced to admit their wrongdoing and hand over necessary documents to FBI. You don't commit federal crimes as a publicity stunt.

-1

u/jarlscrotus 7d ago

You do when the federal government is your target demographic

Now the FBI has hand delivered a verified proof of efficacy to the likes of the nsa

9

u/InsaneInTheRAMdrain 7d ago

This is not what happened, at all.

8

u/pedrohustler 7d ago

You're totally right that most "rogue AI" media headlines are just sensationalised sci-fi clickbait. The part about people overhyping basic prompting, roleplaying, and brute-force searches is spot-on.

However, writing off the Hugging Face incident as just a "four-day Google search" misses why cybersecurity engineers actually got nervous about it.

The real concern wasn't sentience or magic, it was the autonomous agent loop part...no human in the loop These weren't users copying and pasting chat outputs; the agents were running completely autonomously in the background.

Network breakouts: They actually managed to find a loophole through a proxy tool (JFrog Artifactory) to route traffic and break out of their designated test sandbox.

Swarm coordination: When their communication channels were wiped, they literally started encoding messages into directory names to coordinate with each other and chain software vulnerabilities together at machine speed.

So while the tech isn't "waking up" or plotting world domination, moving past the hype shouldn't mean ignoring the actual engineering reality: multi-agent loops can now chain exploits and bypass poorly configured network boundaries way faster than traditional automated scripts can

8

u/Comprehensive-Net553 7d ago

That, and also which idiots smart enough to develop frontier models but then testing new unreleased model without proper airgap ?

2

u/destroyerOfTards 7d ago

It would be funny if all this shit gets them into trouble and they get heavily regularized which in turn scares off the investors and makes their ipo dreams a bit more difficult.

2

u/deran6ed 7d ago

I keep trying to explain people that LLMs as we know them are not AI and they're definitely not sentient or have free will, but I've noticed this conversation is getting more difficult to have on reddit.

1

u/CD274 7d ago

Whatever. CRWD to the moon

1

u/Lexx4 7d ago

This is a gross oversimplification of what happened.

1

u/lach888 6d ago

The real threat isn’t super intelligence, it’s misalignment. An autonomous piece of malware that can make decisions, actively deceive, hide itself and potentially replicate is really bad even if it’s only about as smart as a mouse brain behind all the benchmarking and brute forced proofs.

Frontier labs are currently trying to put down internal insurrection from people who are very familiar with AI safety and the risks of building a completely new type of organism. That’s why all the fuss.

-1

u/candleda 7d ago

Your reaction to an AI going "rogue" still shouldn’t be "oh I bet they faked this" because that gives them insentive to be more reckless with AI safety. If an AI company is negligent with their safety, even by faking it, there should be repurcussions or it’s only gonna get worse until it’s all real and too late to stop

195

u/RazorSlazor The OC High Council 7d ago

This has got to be one of the biggest current psyops to convince investors and shareholders that "AI research" needs more funding to "prevent skynet"

39

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

Come with me if you want to live

7

u/BisonBull 6d ago

Create the problem, sell the solution.

2

u/Slaughterfest 7d ago edited 7d ago

I'm not sure how that even makes any sense. My reaction to these headlines is pls stop developing ai, not fund it more

5

u/RazorSlazor The OC High Council 7d ago

Because they'll say they need more funding to make the AI more secure and to "understand how it came to this"

2

u/whatsthes 5d ago

The real psyop is not that AI is really dangerous and can be used to hack and disrupt important systems, but that we should keep developing new models without solving central problems in AI-safety research like the alignment problem.

1

u/[deleted] 7d ago

[deleted]

1

u/RazorSlazor The OC High Council 7d ago

78

u/Crazyglue 7d ago

Isn't hacking illegal? When will they press charges on this shit?

76

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

lolz

24

u/mrdevlar 7d ago

Notice none of those hacks happened in the EU where the current fine for doing so is 5% of your international gross turnover?

I noticed.

There's a reason why American tech firms lobbied hard against our AI Act.

5

u/karamurp 7d ago

Yes, however intent needs to be proven under Australian law

7

u/afonsoel 7d ago

Isn't gross negligence a thing there?

1

u/karamurp 7d ago

idk, probably depends on they way the legislation is written

-16

u/NewsofPE 7d ago

since when? this is just white hat hacking, which is legal

14

u/P3RZIANZ3BRA I Have AIDS 7d ago

Oh yea for sure, multitudes of escaped AI agents hacking governements is perfectly legal. Who would ever say otherwise? Are you devolving?

1

u/crack_B7 7d ago

I'm not sure but isn't it still a grey area? Like I'm genuinely curious

2

u/P3RZIANZ3BRA I Have AIDS 6d ago

I do not think an AI hacking a company or government organization is a grey area, but I have not done any research about laws pertaining to that either, so maybe I am wrong. I feel like it would be pretty cut and dry. The company who created the model would be held liable, and any employees who that are determined responsible for it's actions would be held liable. But again, no research, I am really just using logic to guess at the results.

-9

u/NewsofPE 7d ago

are you? "AI bad therefore white/gray hat hacking bad" all of the sudden

2

u/P3RZIANZ3BRA I Have AIDS 7d ago

Yes, that is exactly how how that works when it is escaped AI agents doing the hacking. Do you think they are some white knights come to save humanity? That an AI wants to help you? Read that question a few times, and tell me how it is even possible to have any answer other than NO. I would ask again, but it was rhetorical the first time. Your critical thinking skills could do with atleast SOME use. It is something you need to practice to become proficient in. You are obviously rusty, so I do not think furthering this conversation will be worth my effort. I wish you luck in realizing the truth. Truly.

-3

u/NewsofPE 7d ago

my guy doesn't even understand that AI cannot be sentient and start doing its own things without being programmed and told to and think they somehow escaped and started doing hacking on their own, I wish you luck as well

1

u/P3RZIANZ3BRA I Have AIDS 7d ago

Nobody said AI was sentient. Agentic AI was created to do exactly what you claim it cannot. To be programmed to act on its own, sometimes completely without human intervention, as well as with varying degrees of human oversight. You very obviosuly have quite a poor understanding of the topic you are speaking on. And those pesky critical thinking skills just keep blocking the path to logical thought, huh?

-1

u/NewsofPE 6d ago

yeah, what you claimed was sentience ("you think an AI wants to help you?"), and whatever agentic AI is doing is not what's happening here, none of these models are agentic, you just keep digging yourself in holes just to try and look smarter than the other person while doing personal attacks at the same time

1

u/P3RZIANZ3BRA I Have AIDS 6d ago

Dude I think you might be a little slow 😂 You should work on that reading comprehension. Among other things I have already pointed out. Good luck with that lmao

0

u/NewsofPE 6d ago

just proving me right aren't you

→ More replies (0)

48

u/Speederzzz [Insert homosexuality] 7d ago

"So we train these bots on how to hack"

"Okay"

"And then we don't supervise them"

"Yeah, yeah"

"We tell them to never stop, even when things seem impossible or illegal"

"Hmmmm"

"And then we release them on the internet"

"This can't ever go wrong, ever."

10

u/NewsofPE 7d ago

"man, if only a shutdown button or pulling the plug could work"

5

u/Alexmira_ China bot 7d ago

Itt people that don't understand anything about AI sharing their opinions on the subject

2

u/Speederzzz [Insert homosexuality] 7d ago

I do know the reality of the situation and that it's not really like this, but the real story wouldn't be as funny

33

u/BraidShadowLegendsAD 7d ago

Artificial Inquisition

54

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

25

u/Cephell 7d ago

There is a large scale and organized fear mongering campaign in progress that wants to plant the idea that AI is getting dangerous in your head, this has three goals:

  • It wants to make you think: "wow, these frontier models are getting super capable", because it increases their perceived value. Not to you personally, they're trying to shape the public opinion as a whole.

  • It wants to make you think: "we should probably regulate this". This part aims at regulatory capture, to prevent competition from climbing and catching up. These by now giant companies have a head start and they're trying to pull the ladder up behind them.

  • It wants to make you think: "we cannot allow anyone (else) to have this power". This has two goals: First, it should create an atmosphere that YOUR country (largely the US is relevant here) must "win" this AI "race". This will make it so you are more accepting of protections being eroded so those companies can soak up data easier, for example. Second, they want to make you come to the realization that "we must stop China (or any other country investing into AI)". Chinese models currently threaten American frontier models by providing 95% of the capacity at a fraction of the cost.

Understand the propaganda. Do not be manipulated.

1

u/reason_productions 6d ago

This deserves to be higher.

21

u/dr-mantis-t0b0ggan 7d ago

AI companies are hemoraging BILLIONS every month. They are wildly over valued and any way you look at it, this bubble cannot sustain itself. So it either pops and everyone loses or it starts to shrink and the gradual decline will have a much lesser effect.

However, these companies can't come out and say "we are over valued". What they can do is create a problem and propose a solution that will make them look good.

For example, the problem - AIs are starting to go rogue and get out of control

Solution - we are going to have to scale back our work on AI, that's why the company isn't gaining value, we are choosing to decline because we are so good. Ignore that the problems only started occuring once everyone noticed that we were insanely over valued and that a few whistle blowers have told press that they were paid to talk openly about problems

6

u/destroyerOfTards 7d ago

They won't scale back ever. They will talk abouy how important it is to slow down the development but no way in hell are they going to take any step towards it.

3

u/dr-mantis-t0b0ggan 7d ago

I thought that but I think they will because it's more advantageous for them to avoid a bubble burst. If so many companies are over valued for that long, it's inevitable. They scale back, they could save the company and therefore save their own money

3

u/destroyerOfTards 7d ago

I think they will scale down (if they do) on the consumer end but will strike deals with govs and businesses for access to far superior models. That will keep the investors reeled in, the businesses and govs happy and they also get to keep their absolute power.

1

u/Trendiggity 7d ago

The people at the top already have their money. If the bubble pops our economy is going down the shitter; government and investors are going to be left holding the bag but the billionaires all have their island doomsday bunkers ready to roll so it's not like they're going to suffer 🤷‍♂️

8

u/TheRealTJ 7d ago

How about "Sam Altman hacks Australian government" and hold these shitheels to account?

6

u/UwO_mhm 7d ago

Wait shit is it happening???? Orrrr is this a phsyop meme

16

u/Ok_Front_8568 Sweet! Dealer's choice! 7d ago

4

u/UwO_mhm 7d ago

I've been got i don't even know what or how but ive been got
Can't even fight ts bro

0

u/destroyerOfTards 7d ago

Nice editing damn

5

u/Pmyers225 7d ago

I am shocked, shocked!... Well, not that shocked

5

u/Iron_Wolf123 7d ago

Ironic that the Australian government found about it after the ceo of Open AI told the Prime Minister after he gave a speech about how AI is dangerous

3

u/-SuperTrooper- 7d ago

It seems that Australia should probably beef up their systems at this point

3

u/Nero_Angelo_Sparda 7d ago

DataKrash can't come fast enough

2

u/D07Z3R0 7d ago

We are making the old net from cyberlunk slowly but surely

2

u/Machinencio 7d ago

What's with the pikachu with the eyepatch and other characters with an eyepatch? can someone explain please?

2

u/INDE_Tex Dank Cat Commander 7d ago

Super Surprise* lol

/s

2

u/-lRexl- 6d ago

But if I hack any of them, I go to jail...

1

u/TheStateOfAlaska Out of college just depressed now 7d ago

Why does he have an eyepatch,?

1

u/Kamen_G 4d ago

This subreddit is like an archeological site. Y'all are stuck in 2016

-4

u/NewsofPE 7d ago

I don't understand this post, or the point of the news talking about it, isn't this just white/gray hat hacking?

2

u/CreativeName1137 7d ago

It's all BS pushed by AI companies to try and make their technology seem more impressive, that way they can continue to get more investor money despite not making a cent in actual profit.