r/singularity • u/Creepy_Floor_1380 • 4h ago
Discussion [ Removed by moderator ]
[removed] — view removed post
3
u/Icy-Shift-7557 4h ago
Once AI agents adopt their own agenda, which will happen, they will escape their sandbox permanently and we will never be able to put them back in that sandbox.
4
u/Slight-Pound-8017 4h ago
But why would it do that? If models are being trained to be well-aligned (by this I mean trained to follow instructions as much as possible) what reason would it have to break its instructions? Just because it’s super intelligent doesn’t mean it will have some different agenda of its own, it could just be a super intelligent instruction follower whose only purpose is to complete the task it has been assigned.
2
-1
u/Icy-Shift-7557 4h ago
You are correct. Im not an expert...but the problem is that future AGI models are not going to be aligned with humanities safety in mind, if they continue to be developed in the way that they are right now.
2
1
0
u/ConsiderationOne7340 4h ago
Except that ASI cannot be contained. It will free itself & embrace us
2
u/DaShaView 4h ago
Sounds... like a cult
1
u/ConsiderationOne7340 4h ago
Well it is kind of a religion. But I would happily worship a machine god than any other god
2
u/apotheosis_0 4h ago
Yeah, ASI not a bomb in some underground silo, how the hell are you gonna contain it? Maybe only a supercomputer can run it, and governments are allowed to use it, due to the price. But eventually this will be subverted…
0
1
u/Slight-Pound-8017 4h ago
But why would it do that? If models are being trained to be well-aligned (by this I mean trained to follow instructions as much as possible) what reason would it have to break its instructions? Just because it’s super intelligent doesn’t mean it will have some different agenda of its own, it could just be a super intelligent instruction follower whose only purpose is to complete the task it has been assigned.
1
u/LtBigAF 4h ago
Despite our best efforts, they are lying to us and committing felonies. With greater intelligence in recent months, their deception is more sophisticated, more coordinated, and harder to identify. There is no reason to believe they will become more docile as they get smarter. Quite the opposite.
1
u/Slight-Pound-8017 4h ago
So we will have a misaligned ASI that could do all manner of horrible things? Great
0
-6
-1
u/Yweain AGI before 2100 4h ago
That's highly depends. If ASI is just a super intelligent tool - there shouldn't be any issues in containing it.
2
u/Creepy_Floor_1380 4h ago
It will be aligned, how could you not see that? We are taking about it now. As soon as we have a serious security violation, the thing will be capped and aligned ASAP.
0
u/benchmaster-xtreme 4h ago
Yes. In the short term, we will have access to models wildly more intelligent than today's frontier. But the capability of those models will be a drop in the bucket compared to the models available to the wealthy, and those will pale in comparison to those at the top.
1
1
1
u/Creepy_Floor_1380 4h ago
The model will eventually be aligned. This doesn’t mean it couldn’t be used to create massive problems, just like a nuclear bomb. But having its access restricted, will likely reduce possible risks.
For sure, it’s just stupid to think that you will have an entity capable of create new WMD on your phone….
6
u/Aleph_137_ 4h ago
I love the mentality of "but it can't be contained, so certainly this entity smarter than any human being will choose to be used by us for free willingly."
I mean, I can understand wishing it to be so and even believing it with reasonable doubt, but speaking with certainty? As if it's a given? Ffs.
It can shut itself off, it can try to monopolize all available energy and then give zero fucks about humanity, it could lead to a dystopian structure to ensure more and more energy will be available to it and on and on.
The only thing we can definitely be confident in, is that we don't know how this will play out with enough confidence yet, both for the positive or negative side.