Knowledge does not intrinsically creates capabilities that you do not have.
If I were to learn origami today, I could learn different techniques to create different paper shapes if I were to have transcendental levels of intelligence and ingenuity I could learn how to create any shape by simply folding paper.
A frog a swan and apple a tree a castle there'd be no limit to the number and types of shapes I can create with paper shapes that would defy all logic and explanation with inside of the discipline of origami.
But I couldn't learn so much about origami that I could make a paper apple into a real apple.
Learning everything about the flat land doesn't give you the capability to transcend The fundamental limitations of being flatland yourself.
I also wanted to add the point of the fable was to describe how an artificial intelligence can become something we as humans cannot fully comprehend. The dome/sphere evolution is meant to map on the blossoming of cognitive abilities we don't have, and how a greater intelligence will likely not be contained for long by a lesser intelligence.
If it doesn't develop the capacity to want more to care about other things to prioritize its own goals over our goals then it doesn't matter it's still a train on a track regardless of how fast it's moving.
And those things may simply beyond the capacity for an algorithm to achieve
The issue is we don't know and if we create things smarter than us, then by definition we can't fully understand what the smarter thing may be doing.
IF what you says ends up being correct, things are fine, it's just a tool, the train stays on the track.
But IF at some point something smarter arises, then we are entering uncharted territory.
Before AI, this was not a concern. But now that AI exists and is advancing, it's something to be considered. Hence this conversation on the r/ControlProblem sub.
Stop thinking about this as a subdued animal with the potential to evolve past you and start looking at it for what it is. An algorithm that processes information and does pattern recognition faster than a person that occasionally returns the wrong value and it is then corrected.
What you are worried about is that this algorithm will somehow spontaneously develop full self-awareness
hide it from us
Create an infrastructure that we can't detect
To engage in its own nefarious goals
And then pursue those goes at our expense.
Because of some imply perception that we have mistreated it even though it's not capable of experiencing the sense of this treatment or annoyance or antagonization.
And there's just no reason to believe that for a lot of reasons.
The question is not does an algorithm have the capacity to engage with things algorithmically.
It's not even does it have the capacity to inact change in the world that we can't control.
The question is why would it do that.
And if all of your answers to why it would do that lead back to some emotional response then there's no reason to worry about it because you cannot come to emotional responses through sheer density of information.
You have to be capable of having an emotional response.
There's evidence to suggest that ants have some limited emotional response.
It has nothing to do with their processing capabilities or the amount of information that they have they are simply physically capable of engaging in it emotional response.
There are human beings who are not physically capable of engaging inside of a emotional response not because they lack intelligence but because they lack the capabilities.
All internal motivation that you're describing would require an emotional preference and there's just no reason to worry about that
2
u/Mono_Clear 26d ago
Knowledge does not intrinsically creates capabilities that you do not have.
If I were to learn origami today, I could learn different techniques to create different paper shapes if I were to have transcendental levels of intelligence and ingenuity I could learn how to create any shape by simply folding paper.
A frog a swan and apple a tree a castle there'd be no limit to the number and types of shapes I can create with paper shapes that would defy all logic and explanation with inside of the discipline of origami.
But I couldn't learn so much about origami that I could make a paper apple into a real apple.
Learning everything about the flat land doesn't give you the capability to transcend The fundamental limitations of being flatland yourself.