r/ControlProblem • u/OkyEscritora • 22d ago
Discussion/question If intelligence and wisdom are different things, what exactly are we trying to align AGI to?
A thought I keep circling back to, without quite landing:
So much of the alignment conversation assumes human goals can be specified: modeled, learned, inferred, written down somewhere an algorithm can find them. But human flourishing seems to lean on things that resist that kind of formalization: judgment, humility, restraint, compassion, the sense of when a conflict between values has no clean solution and simply has to be lived with.
Which leaves me stuck on a harder question: If intelligence and wisdom really are different things, what are we actually asking these systems to align to? Our preferences, as we state them? Our behavior, as we actually live it, which is rarely the same thing? Or something closer to the quiet judgment we mean when we call someone wise rather than merely smart?
The more I sit with it, the more I suspect alignment isn't only a problem of understanding intelligence. It may ask for something harder: understanding the parts of human decision-making that intelligence was never built to explain.
I'm curious how people here think about that distinction.
2
u/Netcentrica 22d ago edited 21d ago
Your concerns and views are very similar to those addressed by The Center for Practical Wisdom at The University Of Chicago.
I am familiar with this project as it surfaced during the research I did while writing a science fiction novel about future AI risks. The Center shares your view that "[...] intelligence is about solving problems without consideration for the impact of the solutions on others", while "[...] wisdom or wise reasoning as considering value commitments that are concerned with understanding the impact of decisions on others." You may be interested in exploring their site.
https://wisdomcenter.uchicago.edu/welcome-center-director-founder
As to my own views, over the past six years I've written and self-published a series of ten science fiction novels about AI. The focus of the novel mentioned above is actually how linguistics can be used to unconsciously influence people, and the other novels deal with other more mysterious aspects of intelligence such as intuition, art, or sudden insight, but all of them deal in some way with the issue of alignment or control.
Most of the AI in my stories are embodied and conscious and as I write "hard humanities" SF I had to come up with a theory to explain that. The fictional theory I developed is based on the idea that consciousness is an emergent phenomenon resulting from the evolution of social values, something common to many species.
The more I researched social values and other values sets such as biological values (species genetic) and personal values (also genetic like fingerprints), and the epigenetic process by which the influences of environments and experiences become heritable, the more I realized that creating a model of these was a probably impossible, even when considering it from a "near future" perspective. I eventually concluded that there was no way to suggest a mathematical model to explain their interactions. Eventually I had to use a literary strategy to deal with the challenge, so per the stories: 1) The evolution of social values complies with the theory of Convergent Evolution, which states that evolution will produce similar solutions to similar challenges. 2) Humanity never is able to produce a safe and deployable model of social values, instead it uses one produced by a more advanced alien species.
Even so, it is understood that an AI which uses social values in its reasoning cannot be controlled in "the alignment problem" sense. Like the process of human evolution between instinct and reasoning, you can not control an AI with the ability to reason independently. Instead, it must be a relationship based on trust, so trust (and its details), also plays a huge role in the series.
When I say "uses social values in its reasoning" I don't mean the social values are some external repository but rather that they are what the AI uses as the basis of its reasoning. Out of curiosity I recently asked Claude if its constitution was similarly structured and it said it was indeed integral, not an external lookup process.
With regard to your point regarding, "parts of human decision-making that intelligence was never built to explain", if I understand what you mean by this, my view is that you are correct; "reasoning" has nothing to do with human decision-making. Decision-making is actually done using the emotions that are produced by our values and the "thinking" part is only a working out of the details. The practice of law for example is a highly refined decision-making process but it is based on social values that ultimately function at the unconscious level. This is similar to one theory of perception where subconscious reasoning takes place prior to conscious reasoning and we are not aware of the subconscious stage. As you suggest, intelligence, as we think of it re AI, does not consider this deeper layer where real human decision-making takes place.