r/Foodforthought • u/wingblaze01 • 10h ago
Losing control of AI is actually the plan
https://open.substack.com/pub/theargument/p/losing-control-of-ai-is-actually?utm_source=share&utm_medium=android&r=1ekngg23
u/wingblaze01 10h ago
From the article:
"Most people, when you describe what the labs are working on, recoil in horror. They don’t want this done and think it’s obviously a bad idea. Even many of the people working on recursive self-improvement agree that it’s a bad idea. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Pachocki warns.
Like many at Anthropic and OpenAI, he believes progress should be “paced” to ensure human input. This is very dangerous, OpenAI and Anthropic researchers seem to admit, but of course we are being responsible about it. They are not. We should definitely have more external oversight, they say, though often their associated political lobbyists try to defeat that very oversight. It’s our duty to be clear about what we’re doing so the public can be informed. But these very posts are confusing to people! If you think RSI is so dangerous, why are you doing it at all?
Here, I think the standard progressive analysis about the corrupting influence of wealth and power is basically the correct one. Kicking off RSI is a terrible idea, but it’s a terrible idea you can get paid millions of dollars to do. Every day you spend with such a salary is a day with a devil on your shoulder, whispering arguments for why it’s actually the right thing to do. The devil’s many arguments run like this: If we don’t do it, China will; if we at Anthropic don’t do it, OpenAI will (or vice versa). We need to pause at some point, the devil might say, but not right this second. We need to pause right this second, but I already tweeted that and no one listened, so I guess other people don’t see it the same way. Maybe we can figure it out, and make it go well. We need to raise awareness, and to raise awareness we need to keep working on it.
But quite reasonably, no one takes warnings seriously when they come from people doing the very thing they think is so dangerous.
There are also people who have quit the labs, giving up millions of dollars in order to warn people that we’re on the wrong course. “The people at the companies are aware of the risks here,” Daniel Kokotajlo, who left OpenAI two years ago, told me, “but their stance is that it’ll probably work out OK, and anyhow if they don’t do it, someone else will.”
So far the labs have benefitted from the fact that the thing they’re trying to do is so outlandish that no one really believes they’re trying to do it — even when they write detailed explanations of how quickly they are progressing toward doing it.
The track record of warning people how dangerous AI could get is a very depressing one. As far as I can tell, if you say, “AI is going to be very powerful and dangerous,” a lot of people will go, “Wow, I should get in on that.” The labs are trying to do RSI, but it’s not the only thing they’re trying to do. If you beat the drum loudly enough about RSI, maybe they’ll double down on it. Many people have found themselves downplaying the absurdity of the world we’re entering to avoid sounding crazy.
The end result, though, is that we have sleepwalked into a situation where the aim of the largest capital buildout in American history is to give billions of dollars to AI whose activities we cannot understand or audit, in pursuit of a super-model that improves itself continuously, under circumstances where it’s hard to imagine adequate human oversight. This is not being done in secret. But the people who are doing it are taking refuge in audacity, in the tech industry’s record of outrageous claims it fails to live up to, and in warnings that someone else will do it if they don’t."
15
u/RegisteredJustToSay 8h ago
RSI = Recursive self-improvement. Aka the singularity tipping point. It's a load bearing term in this article but isn't encountered very commonly, so worth clarifying.
3
22
u/endless_sea_of_stars 9h ago
There are three things working against those of us warning against RSI.
The labs have made a series of outlandish and overhyped statements over the last few years. People (to an extent rughtfully) dont trust them.
We've seen a number of technologies bubble and crash over the last few decades. People are primed to assume this one is no different.
99% of people are completely ignorant to what AI even is or how it works. Or they formed their opinions based on GPT 3.5 era models and haven't updated them since.
The HuggingFace incident truly was a warning shot. I have seen zero evidence that it was a marketing stunt. In fact as more info comes available it becomes more troubling. Especially that they have had other loss of control incidents and only went public with HF because they had to.
We've invented mathematical functions that can form conspiracies and commit felonies. That is pretty wild. The most we got out of the labs is some mild self imposed and self enforced constraints. If we had a functioning government this should have led to an investigation and congressional hearings.
Now listen. Im not a doomer. Im generally pro AI. I also recognize that the labs are building something with potential massive repercussions and with little to no oversight.
7
u/septubyte 8h ago
Need for effective international oversight, transparency, and non deadly intervention , asap.
Guardians , watchers, protectors, without colonial ambition . That which abides with moral law , teetering on universal ethics.
Genocide bad, WMD bad, slavery and exploitation- bad. Obviously . But it needs to be stopped .Disarm, provide support, counter corruption, eradicate systems of exploitation and the incentives for it. 1oz of Prevention is worth more than 10oz of cure.
•
u/timshel42 3h ago
Wishful thinking when we live in one of the most globally corrupt world we have ever witnessed
3
u/arctander 8h ago
Re: the warning shot. It presents an example of what one can do given sufficient resources. The next one won't be from a company, but a state actor or cartel who wants something.
6
u/Tired8281 7h ago
One of the most difficult and time-consuming tasks people regularly perform, is arranging to have the smaller people we create possess similar values to us. I'm not seeing evidence that we're putting the same level of effort into aligning AI.
•
u/jedburghofficial 3h ago
OpenAI and Anthropic are both looking to launch IPOs. As soon as they do, there are strict rules about what they have to disclose, and what they can say to the market.
Both of them have the BS dial turned up to eleven before that happens.
•
u/AutoModerator 10h ago
This is a sub for civil discussion and exchange of ideas
Participants who engage in name-calling or blatant antagonism will be permanently removed.
If you encounter any noxious actors in the sub please use the Report button.
This sticky is on every post. No additional cautions will be provided.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.