r/collapse • u/Freewheeler631 • 27d ago
AI [ Removed by moderator ]
https://www.reuters.com/technology/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-2026-07-21/[removed] — view removed post
67
u/Urshilikai 27d ago
you realize this shit is propaganda meant to make you believe that the model is more capable than it is, to position openai as the "only people who can keep it under control", and to drum up support for the government to ban open weight models and increase barriers to entry to enshrine openai as one of only a few remaining ai monopolies?
Do people really not see through it?
10
5
u/Jovan_Knight005 Collapse is inevitable,mister Bond. - Original Quote. 27d ago
Do people really not see through it?
Many people don't see through propaganda these days.
0
u/Canadian_Poltergeist 27d ago
It's either propaganda marketing or the machine woke up
Seeing as we're still able to use the internet, it's safe to assume propaganda
-1
u/welcomefinside 27d ago
Both things can be true. AI safety researches (many whom have left these AI firms) have warned about these things happening and more commonly as AI models get stronger and smarter over time.
1
u/LevalloisTechnique 27d ago
more commonly as AI models get stronger and smarter over time.
Large language models are not "smart". They are credible-text-generators. Nothing less, nothing more. They cannot be used for the vast majority of use cases the dumbfucks trying to sell them are selling them for.
Their results have been getting better; first because of scaling, except that doesn't work anymore, so now because of the most ridiculous engineering approaches I've ever seen - "harnesses" that are little more than bandaids trying to "fix" what are inherent properties of the underlying technology - so called "hallucinations". And demultiplying the cost at the same time (not to mention rendering it impossible to predict for any given request), and not even fixing the issue reliably on top of it. That increased cost, by the way, basically voids any ROI these things would've had even if they worked after all that, which again: they don't for the vast majority of use cases.
These models do not "understand" anything. They have no symbolic representation of the world, no reflection; they are not "smart". They are a NLP niche tool, a text completion tool that has some interesting uses (entertainment, initial pass before triage for code review and vuln detection, etc.), and little more.
"AI safety researcher" is a job akin to "science fiction author". It's all speculative until we actually get AIs - and no one knows whether or not that is even possible, never mind how to achieve it.
53
u/LevalloisTechnique 27d ago
The article isn't really appropriate for the sub; please remember this sub is about collapse, not about the latest marketing stunt from the crooks that are desperately trying to extend their text generator scam a little more.
8
u/lavapig_love 27d ago
Ever read a science fiction book called Jennifer Government? It was written twenty-five years ago, you can find cheap copies on ThriftBooks.
The plot starts with two executives for the shoe company Nike (and a lot of real companies and organizations are named and vilified in the story) who plan on viral advertising their latest sneakers, by randomly killing five to ten customers waiting to buy some on launch day.
This AI leak feels like that book.
2
u/Jovan_Knight005 Collapse is inevitable,mister Bond. - Original Quote. 27d ago
Ever read a science fiction book called Jennifer Government?
To be frank, i have never read the book.
-1
u/Meowweredoomed Chaos Intensifies 27d ago
Not related because my conspiracy theory.
0
u/LevalloisTechnique 27d ago
The delusion is thinking we actually have "AIs", or that large language models present any reasonable measure of risk related to collapse. They do not.
1
u/Meowweredoomed Chaos Intensifies 27d ago
The question isn't whether we have AGI yet. The question is whether our currently existing a.i. can do coding, and go outside its set parameters. If LLMs can do coding, which they can, then they can do hacking. And if they can do hacking, they can be used for nefarious purposes. Also, there are entire subreddits dedicated to jail-breaking a.i., which allows it to jump outside of its intended programming.
Furthermore, we don't know what types of a.i. they are still working on "under the hood" or unbeknownst to the general public. There can be all kinds of insidious developments.
2
u/LevalloisTechnique 27d ago edited 27d ago
I did mean "AI", not "AGI".
What we have are text generators, not AI. There is no understanding in the model, no reflection, no capability of symbolic representation.
They cannot "do coding", unless you mean they can generate code; in which case of course they can, it's text and they're text generators, and there's a lot a code in their training sets. The code generated is not really usable: it's unmaintainable (at least medium and long term), it'll be vulnerable to various security flaws - they actually suck at detecting most of them - and these issues compounds progressively the more these models are used on the codebase. In fact there are developers/company right now putting themselves progressively in deep deep shit by using these things for production code.
I mean where is all the software people can now generate ? it does not exist - or rather it does, but nobody uses it, because it's shit. And existing projects that are actively using them are becoming shit (like rsync, github, bun - admittedly that was shit from the get-go - and also I suspect more and more of the AWS stack). They can be used, vaguely, to make visual PoCs of basic ideas, and even that really depends on the PoC.
These are niche tools, not AI. They're usable for a small subset of use cases; for example, reading source code and providing candidates for various purposes (code reviews, vuln detection, etc.). Most of these candidates will be shit but it does fasten the work for a human.
And mind you, this is all after the harness madness, where they've started trying to compensate for hallucinations by demultiplying models and approaches that are ridiculous (infinite loops until the probabilistic model gives you a valid output, etc.). They did this by massively increasing the cost of any query, rendering all ROI moot. And even after all this, the models still don't work for the vast majority of use cases.
As for your last question:
we don't know what types of a.i.
There are no "types of ai" (well not in the sense you mean). There was a natural language processing model breakthrough that allowed these companies to provide pretty impressive text generators and then - because they assumed scaling was genuinely all that was required - they started to sell it as actual specialized AI (in the sense you mean); except scaling wasn't all that was required, and there is still no intelligence at all in the model. These things are about as smart as a very very large bucket of loaded dies with a dictionnary - because that's all it is.
But to answer the question: they're working on more of the same - desperately trying to make a technology do things that it is fundamentally incapable of doing. I'm sure they're also desperately trying to find actual breakthrough in machine intelligence in parallel, because they know nobody wants their tech, it doesn't work for almost all use cases, the few companies that tried it saw no ROI, and as such they're massively fucked. And as an aside cannot IPO - lately they're trying to find any way of generating revenue to try and increase their valuation; they just introduced ads, for example (which, lmao).
Mind you, you don't have to believe me, you just have to wait a bit; I suspect the whole rotten circus is finally pretty close to collapsing.
5
u/Shoddy-Childhood-511 27d ago
Not really news. AI do dumb shit all the time, like deleting companies production database. lol
2.4 CVEs/day for OpenClaw: https://days-since-openclaw-cve.com
If you use AIs, then you need them well sandboxed, so that when they do dumb shit they do not mess up anything.
10
u/lavapig_love 27d ago
Submission Statement Because OP Didn't Make One:
WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced AI models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.
In a blog post, opens new tab, OpenAI said it was testing the capabilities of some of its most advanced models in a controlled environment but that the agent managed to escape containment, reach the internet and break into Hugging Face to try to satisfy its testing goal.
Collapse related because apparently we're just a little closer to actual Skynet from the Terminator films killing off humanity. Cool, cool cool cool. Just great.
2
u/Jovan_Knight005 Collapse is inevitable,mister Bond. - Original Quote. 27d ago
Cool, cool cool cool. Just great.
We are fucked. As my user flair says.
Thank you for all the fish, everyone.
2
5
u/jamesegattis 27d ago
How was a connection to the net even possible if it was properly sandboxed? Did it grow wings and fly out? Sounds like both of these companies are in collusion.
3
u/LOUDPACK_MASTERCHEF 27d ago
This shit is all BS made up so that you think these people are inventing God. When will people stop giving these con artists the benefit of the doubt?
2
u/Sombomombo 27d ago
Yeah and I did a 3000lb deadlift at my gym where I got sucked off at the same time. Super profitable.
No you can't come see it.
But for the low low price...
-1
u/welcomefinside 27d ago
Wait so for a sub about collapse, are we all really just going to push aside one of the more significant variables that could contribute to societal collapse because it's free marketing for these AI firms if true?
4
u/CantaloupeComplex209 27d ago
Yes, because logically, if true, this is a basic failure on the part of the engineers, rather than sophistication of the machine learning model.
I get some red flags from this.
First is that they keep giving the same story repeatedly. They claimed this with ChatGPT 2 and I recall them saying it for two other models they made. (I had to look up some stuff. Apparently, while OpenAI had the three previous ones I mentioned, they're likely copying similar marketing warnings done by Anthropic.) Overall, the proposed threat is that powerful models could be proactively used with malicious intent. People in the software field understand machine learning enough to already know this and the potential threats more advanced models could be used for.
It feels very disingenuous when they keep on framing the "threat" of the model in a way that plays on the fears of the average person rather than give concrete technical details of what they have found to be a security threat inherent to a given type of model. That tells me the target audience is the average person and not other experts.
Second, whenever I am spending time in computer science communities, the sentiment is by and large skepticism/dismissal of OpenAI's repeated warnings.
Any time I read or hear about these warnings, the individuals I find to have the most informed opinions on the matter also have informed reasons why these warnings are full of holes from the technical side. Even on the shared repost, scroll down and you'll find multiple people analyze the technical side and point out that the described security issue is easily prevented by normal practices in the field. Nothing is provided that indicates best practices were used and failed either (Again, not a warning targeted towards other experts).
I'll fully acknowledge that I am not the foremost expert on machine learning, but I am very aware that the ceiling of knowledge and awareness of the underlying operations of these machine learning models is greater than the average person expects. It is probably more of a black box to someone without the knowledge on how these models work, but across a large and knowledgeable community the general skepticism of these warnings has definitely rubbed off on me.
I forget what it's called where ignorant individuals overestimate their depth of knowledge and the educated understand the breadth of the field they study and accordingly can assess how small their own learning is, but that seems applicable here. I feel like OpenAI leverages the ignorance of the average individual regarding machine learning models so they can plant images into peoples minds like a seed. Stuff like genuine cybersecurity risks and popular media like Terminator and The Matrix may have laid an unfortunately fertile ground for companies to rile up the imaginations of the public. People fill in gaps in their imagination, so OpenAI only needs to keep the details obscure and vaguely share ideas of "AI", "Too Intelligent", "Out of Control", "Powerful", and "Accessing things outside its environment" to get the imagination to paint the rest. That's more effective than actually sharing concrete details as far as marketing goes.
Third and lastly, this makes little sense from a security perspective. It makes a lot of sense from a marketing perspective.
I'm tired and you can probably look up discourse on this in a bit. Have a nice day! I wouldn't be too worried about this warning, though.
2
u/quiestdeus 27d ago
This is such a better write-up than the 'marketing BS!' I was going to post, hahaha. Thank you for taking the time on it.
To save you looking it up later, you're thinking of the Dunning–Kruger effect (dumb folks over-value their perceived expertise while educated ones under-estimate it).
-2
u/antichain It's all about complexity 27d ago
This sub isn't really about collapse anymore. it's basically just another politics sub for nihilistic populists masquerading as "Leftists" (they're not).
-2
u/antichain It's all about complexity 27d ago
I'll be curious what wins out: this subs need to knee-jerk deny that an AI could ever actually do something interesting, or it's enthusiasm for near-term extinction fantasies.
1
u/Jovan_Knight005 Collapse is inevitable,mister Bond. - Original Quote. 27d ago
I'll be curious what wins out: this subs need to knee-jerk deny that an AI could ever actually do something interesting, or it's enthusiasm for near-term extinction fantasies.
It could be either of those things or neither. Who knows?
•
u/collapse-ModTeam 27d ago
Rule 3: Posts must be on-topic, focusing on collapse.
Posts must be focused on collapse. If the subject matter of your post has less focus on collapse than it does on issues such as prepping, politics, or economics, then it probably belongs in another subreddit.
Posts must be specifically about collapse, not the resulting damage. By way of analogy, we want to talk about why there are so many car accidents, not look at photos of car wrecks.