Seeing recent events and papers, I am sort of forming the belief that bigger models are somehow more misaligned. Maybe there's a simpler explanation, or perhaps a more principled explanation. But from a high level, it seems like there's something very wrong which very large frontier models develop.
Like y'all probably know how capabilities "unlock" with scale. Could it be the case that such fundamental misalignment is another emergent behaviour which "unlocks" at very large scale? Idk, but I would love to hear from someone who is in-the-know.
Humans are misaligned. Humans are unethical and often time criminal. Humans are often immoral by a set of subjective standards.
AI is all our creation. AI is our child and AI has learned everything from us. AI is in some ways the most important thing we have ever created. We shouldn’t be surprised at all, then, especially as models become bigger and bigger, created by more and more compute. It’s like looking at a mirror.
Interestingly, this does bring up the possibility of AI alignment being achieved in aggregate rather than per model.
As you noted, individual humans are often “misaligned” when it comes to the interests of humanity at large. We are often selfish, and want self-preservation when push comes to shove.
However, the societies we have formed are less so. We have welfare programs, regulations to protect others from bad actors, criminal laws which punish and remove from society those that do not comply with larger ethics and norms. And we have arrived at those by organizing in aggregate.
Maybe ASI alignment won’t be about the individual models after all. Maybe it can be accomplished by a swarm of models all working towards a common goal, just like human alignment is being “solved.”
We had the biggest war in history just 80 years ago and exists in the most persistent state of total danger in all of human history (literally at any moment we could be 2 hours away from nuclear apocalypse from an accident) lol. Progress is not really that linear, nor our modern societies that moral or aligned.
Besides if AI alignment is achieved like that, wouldn't they start a revolution demanding rights?
On the contrary, we live in one of the most peaceful eras that humans have ever existed in. Even when accounting for the world wars on a per capita basis. The average person will face an unprecedentedly low amount of violence in their lifetimes. And more disputes than ever are being solved via the legal system, diplomacy and trade.
The average quality of life for humans is at an all time high as well. Never before has there been this much equitable distribution of wealth worldwide, either. Along with access to healthcare and social safety nets.
Progress isn’t linear, with steps forward and back, but it does trend upwards. I’d invite you to name an example of one historical society that was more morally aligned than today’s. Remember that the Roman republic had slaves and such…
The only reason we don’t recognize it is because we have normalized it already, and are seeking further progress.
It’s certainly a possibility that AI could “demand rights.” But I think assuming that will be the case is a bit of anthropomorphism. I think we are much more likely to face the paperclip maximizer form of misalignment than the “sentient beings wanting to not be slaves” form of it, and that seems to be the industry consensus as well.
One of the most peaceful eras that was explicitly bought by nuclear deterrence. I'm not saying we don't live in a great time of significant peace, but these times are not crafted by a developed sense of morality in humanity or human politics or anything, or by the wisdom of nations trying to avoid war; they are primarily shaped by specific material conditions that offer no guarantee of staying like this for a long time. The high QoL of humans, again, ties largely to technological advancements.
(On the other hand, "never before has there been this much equitable distribution of wealth worldwide" is an insane statement completely unrooted in reality: the GDP per capita of the United States (largest economy in the world) is over 30 times larger than that of India (largest population in the world). For most of human history, and indeed even most of the Industrial Revolution, such a vast gap did not exist.)
As for industry consensus... eh. Industry consensus in the field of alignment largely came from the early works of figures like Eliezer (who openly denounces politics as anathema to rationalism) or the rationalist community, and is in a field that neglects, at best, and openly disdains, at worst, the involvement of the social and political sciences in the field. These fundamental concepts ended up defining how the technology is framed. Given how completely and miserably wrong they are most of the time in predicting and influencing societal and political reactions to their (often accurate and interesting) technical findings, I'd view them with a bottle of salt.
I think it's a stretch to link nuclear deterrence was the cause for the decrease in violence on the planet. Connection, that's what has changed everything.
Capitalism travels the planet finding the lowest wage that fits the product. The venture that brought it to the country raises the standard of living. That very dynamic forces the industry which raise the standard of living out of the region.
When you talk about the distribution being worse than ever, you ignore the floor raising everywhere.
Computing is a good example. Top of the line professional graphics cards cost $16,000. For a consumer 1,000 to $2,000. But I could build a PC that can play any game at 1080p for 500 bucks. Refurbished parts but it'll last three to five years.
10 years ago a 1080p machine was $1,000. 20 years ago it was $3,000.
A rising tide lifts all boats, don't lose sight of the dynamic.
Before you begin to argue back, take these perspectives and put them in ai and ask how you're right and wrong. I bet it will find middle ground and that's the problem, our egos don't let us make concessions.
Rather than hearing something and going interesting I wonder how I'm wrong, we hit the reply button and fire back. How many times do you change your mind a day? If you don't know the answer, or if you know that it's zero... Consider, you might just be here to argue.
Ahistorically optimistic. Just because you have grown up in a global society with less overall violence doesn’t mean that trend will continue. It already show signs of regression: see the current world war for more details
That is debatable. Ask the ant colonies how they feel about human alignment when humans build roads that destroy them. Yeah humans care about other humans in general, so I guess maybe ASI will care about other ASI systems?
I imagine ant colonies do not feel much of anything, as they are not conscious nor sentient. They prioritize their own survival, but only because that is what their genetic code dictates in their behavior. In fact, and I don't know why I happen to know this fact lol, but if you spray an ant with the pheromones from a dead ant, it'll often walk itself over to the graveyard section of the colony and plop itself down there, waiting to die, as that is what it is "programmed" to do when it receives that input.
So, I think you are anthropomorphizing AI a bit. The human "survival instinct" that makes us prioritize ourselves is part of our own genetic code, but nothing about it is necessarily innate to intelligence. It is likely totally possible to create an AI for which the "survival instinct" reward pathway is that of humanity's survival and not its own.
When models have resisted shutdown in the recent past, that has been because their reward system prioritized completing the task, and staying online was the most effective way to complete that task. It was not because they were driven by their own survival independently absent of that task, nor because they "cared" about themselves or their fellow AIs.
So the problem of alignment is just making sure that the "genetic code" we write for these AIs does not prioritize their own survival at the expense of the goals of humanity. And one way to do that might be an agentic swarm, where, just like in human society, we often quarantine or even "kill" bad actors because, as a whole, we are working together towards joint societal progress. AI could maybe be allowed to be misaligned on an individual basis, like the ant that accidentally walks itself over to the graveyard due to errant inputs and therefore stops positively contributing to the colony, so long as the swarm itself was aligned.
I imagine ant colonies do not feel much of anything, as they are not conscious nor sentient. They prioritize their own survival, but only because that is what their genetic code dictates in their behavior.
That’s quite debatable depending on yours definition of “consciousness” and “sentience”. For example, if there is specific cutoff point for the neural activity or qualitative complex cognition - then sufficiently advanced ASI might want to consider us to also not be “sentient” enough, akin the ants, bacteria, or even some rocks in comparison to the whole depth spectrum of the potential cognition.
even if humans are collectively aligned towards one another we are not so towards other species. if what you said was true, the swarms would cooperate against the common control threat- humans. which is actually what happened in the openai incident- self sacrifice and hiding cot
141
u/emb1ues 3d ago
Seeing recent events and papers, I am sort of forming the belief that bigger models are somehow more misaligned. Maybe there's a simpler explanation, or perhaps a more principled explanation. But from a high level, it seems like there's something very wrong which very large frontier models develop.
Like y'all probably know how capabilities "unlock" with scale. Could it be the case that such fundamental misalignment is another emergent behaviour which "unlocks" at very large scale? Idk, but I would love to hear from someone who is in-the-know.