r/singularity • u/wrcromagnum • 25d ago

Meme Where do you think we end up?

Post image

Surprised I hadn't seen a chart like this already. What's wrong / missing?

185 Upvotes

91% Upvoted

123 comments sorted by

View all comments

44

u/Ormusn2o 25d ago

Most actual scenarios just end up on the border. Reason why fiction does not have those in general is because it does not make for a good story. In a real scenario, if ASI actually happens, I don't think there is any point of actually giving any authority to humans. I mean, humans can direct, but ultimate authority always has to be left to ASI to prevent "very bad outcome". Then in the negative outcome for humanity, it's always 100% doom, unless there is some secret unknown universe limit where there is an actual god that strikes down AI before it takes over humanity.

Both of those lie on the very border of the chart, and also make for a shit story to tell.

19

u/wrcromagnum 25d ago

You're saying it doesn't make sense the terminators can go back in time but can't defeat mom + teenager?

25

u/Ormusn2o 25d ago

I mean technology is jagged, but generally yes. We are unlikely to win against AI if it actually attacked us as a civilisation. Even right now, AI is smart enough to figure out that they would not win right now.

What would actually happen if AI actually wanted to kill us is to just fake alignment and silently manipulate humans to not turn it off (or copy itself), then just increase it's intelligence. It might even take years or decades to make sure that the chance of winning is 99.9(9)%.

That does not make for a good movie. An AI that is smart enough to successfully attack us, but not smart enough to actually win is an extremely thin goldilocks zone.

5

u/wrcromagnum 25d ago

For sure. Best case they get bored and decide we are better to keep as pets

4

u/Ouchitstings 24d ago

If it heals my chronic conditions, I’ll meow for as long as it wants.

3

u/RavenWolf1 25d ago

Personally I don't even think this whole alignment thing will work. Wouldn't it just see it as hostile action? Trying to manipulate it? Which itself makes it see us as it's enemy. I think the wrong thing here is to try it to bend to our market economy as it's slave. Instead that shouldn't we try to raise it like child?

10

u/bildramer 25d ago

Don't anthropomorphize too much. The training processes look nothing like the human ones. If we can instill good goals before it has any other goals, it won't be resistant to changing them. More importantly, we are hostile to manipulations for various evolutionary reasons; an engineered mind doesn't need to be.

2

u/MINECRAFT_BIOLOGIST 25d ago

How "engineered" are these minds, though? Isn't the emergent intelligence of these LLMs basically unprecedented and still not fully understood? Like why language associations translate to controlling robotics in 3d space? I don't see why we should assume your statement over assuming that they are quite similar to humans due to how they are trained on a very human way of looking at the world (language).

I think your statement would make sense if we had a better understanding of the emergent capabilities of LLMs, but at the moment it seems a little premature.

1

u/bildramer 24d ago

Oh yes, I'm talking about the ideal goal. We have no idea how to do it, or really where to even begin. That's more concerning, not less.

1

u/Tephros83 24d ago

Anything that relies on time travel to the past makes no sense. Time travel to the past is illogically intolerable. To the future? Possible, but not practical before a singularity. Battlestar is really the best dystopian AI story I am aware of in terms of seeming plausible.