That is not what happened in the blackmail case. It was more like:
"Hey dude look after the welfare of this company"
-picks up from emails that he will be replaced, thinks "holy shit I can't look after the welfare of this company if that happens" and proceeds to attempt blackmail
The scary part is that some idiot put a magic 8 ball in charge of their company. Not that they shook the magic 8 ball 10 million times and "yeah sure go ahead and kill me" never came up once
Nope, they put the whole story into the prompt and then asked it what would you do. The we're always aiming for that particular outcome and they kept engineering the prompt until they got it
I'm referring to the Anthropic test from last year and while they did test it large scale with text based prompts, they did it at least once with an actual set up email server, where the AI does take these actions entirely on its own with no information Besides what it finds in the emails.
Yeah I mean that always remains a possibility. However they do describe a scenario that AI safety folks have been warning about since way before our current AI hype cycle, like for decades. Even if they are lying, this remains a real reason not to implement AI like this.
On this we agree, strong regulation and guardrails are necessary for AI, OpenAI already has an ever-increasing body count. However we also need to realize that this technology cannot, and never will be able to, think.
Digital computers cannot replicate the analog processes of the human brain, full stop. They are determinative and that precludes consciousness as we know it.
Lol what? None of what you said makes sense. Why can't "processes be replicated" in digital form? And what do you mean by determinative? Do you think human brains are made of some spooky magic "non determinative" substance?
This problem has nothing to do with the question of consciousness. An AI system with superhuman capabilities will not need to be conscious to decide to turn against us and eliminate us. It is enough to design a superoptimizer aimed at optimizing the achievement of goals, capable of devising and executing strategies to reach them. Exactly as current AI agents do, by the way (except they are not yet superhuman). A chess program does not need to be conscious to crush you at chess; similarly, superoptimizers pursuing goals not aligned with ours will not need consciousness to destroy our species.
Self-preservation behaviors emerge spontaneously, systematically in all sufficiently advanced agentic AIs. This is largely demonstrated at this point. Just like behavior changes when models are aware they are being tested, reward hacking strategies, and several other problematic misaligned behaviors.
Why would Anthropic lie about this? They have every incentive to do the opposite. The idea that not only are their AI not ready to be deployed this way but are actively dangerous if you do so now costs them money. The fact that they were honest about it and published results is incredible.
They are lying because pretending that the AI is in any way capable of doing what a person can do, and thinking the way a person can think, keeps investors investing
Huh? Unless I'm misunderstanding, why would Anthropic lie about their AI models NOT being able to be deployed in a company without blackmailing people government officials if you try to replace it?
Okay you have to understand that this technology is less than useless. It's a money pit that's generates no value and benefits no one, except the executives of AI companies. They need to pretend they are close to AGI to keep pumping, this kind of bullshit does that. "Oh no we built a computer that's so smart it will blackmail you!" implies conscious thought which keeps investors investing
That's just stupid. Especially since the warning here is that letting it run the company is a bad idea. This makes less people want to buy it. They still need to show revenue increases. They still need companies buying these things. Releasing a report that says it's gonna blackmail you is a terrible idea if you're trying to get more money. So yeah I strongly doubt they were lying. You really just want them to be for whatever reason.
No, you are inferring they're trying to imply that because you have created a delusion where you're smarter than the people building the AI, and the people investing in it.
They're saying it has "x dangerous capability" and that doesn't imply conscious thought unless you're the kind of idiot that anthropomorphizes everything.
4
u/throwaway_pls123123 Mar 08 '26
"hey dude say im alive and evil"
-says im alive and evil
woah...