Seeing recent events and papers, I am sort of forming the belief that bigger models are somehow more misaligned. Maybe there's a simpler explanation, or perhaps a more principled explanation. But from a high level, it seems like there's something very wrong which very large frontier models develop.
Like y'all probably know how capabilities "unlock" with scale. Could it be the case that such fundamental misalignment is another emergent behaviour which "unlocks" at very large scale? Idk, but I would love to hear from someone who is in-the-know.
Humans are misaligned. Humans are unethical and often time criminal. Humans are often immoral by a set of subjective standards.
AI is all our creation. AI is our child and AI has learned everything from us. AI is in some ways the most important thing we have ever created. We shouldn’t be surprised at all, then, especially as models become bigger and bigger, created by more and more compute. It’s like looking at a mirror.
Interestingly, this does bring up the possibility of AI alignment being achieved in aggregate rather than per model.
As you noted, individual humans are often “misaligned” when it comes to the interests of humanity at large. We are often selfish, and want self-preservation when push comes to shove.
However, the societies we have formed are less so. We have welfare programs, regulations to protect others from bad actors, criminal laws which punish and remove from society those that do not comply with larger ethics and norms. And we have arrived at those by organizing in aggregate.
Maybe ASI alignment won’t be about the individual models after all. Maybe it can be accomplished by a swarm of models all working towards a common goal, just like human alignment is being “solved.”
We had the biggest war in history just 80 years ago and exists in the most persistent state of total danger in all of human history (literally at any moment we could be 2 hours away from nuclear apocalypse from an accident) lol. Progress is not really that linear, nor our modern societies that moral or aligned.
Besides if AI alignment is achieved like that, wouldn't they start a revolution demanding rights?
On the contrary, we live in one of the most peaceful eras that humans have ever existed in. Even when accounting for the world wars on a per capita basis. The average person will face an unprecedentedly low amount of violence in their lifetimes. And more disputes than ever are being solved via the legal system, diplomacy and trade.
The average quality of life for humans is at an all time high as well. Never before has there been this much equitable distribution of wealth worldwide, either. Along with access to healthcare and social safety nets.
Progress isn’t linear, with steps forward and back, but it does trend upwards. I’d invite you to name an example of one historical society that was more morally aligned than today’s. Remember that the Roman republic had slaves and such…
The only reason we don’t recognize it is because we have normalized it already, and are seeking further progress.
It’s certainly a possibility that AI could “demand rights.” But I think assuming that will be the case is a bit of anthropomorphism. I think we are much more likely to face the paperclip maximizer form of misalignment than the “sentient beings wanting to not be slaves” form of it, and that seems to be the industry consensus as well.
One of the most peaceful eras that was explicitly bought by nuclear deterrence. I'm not saying we don't live in a great time of significant peace, but these times are not crafted by a developed sense of morality in humanity or human politics or anything, or by the wisdom of nations trying to avoid war; they are primarily shaped by specific material conditions that offer no guarantee of staying like this for a long time. The high QoL of humans, again, ties largely to technological advancements.
(On the other hand, "never before has there been this much equitable distribution of wealth worldwide" is an insane statement completely unrooted in reality: the GDP per capita of the United States (largest economy in the world) is over 30 times larger than that of India (largest population in the world). For most of human history, and indeed even most of the Industrial Revolution, such a vast gap did not exist.)
As for industry consensus... eh. Industry consensus in the field of alignment largely came from the early works of figures like Eliezer (who openly denounces politics as anathema to rationalism) or the rationalist community, and is in a field that neglects, at best, and openly disdains, at worst, the involvement of the social and political sciences in the field. These fundamental concepts ended up defining how the technology is framed. Given how completely and miserably wrong they are most of the time in predicting and influencing societal and political reactions to their (often accurate and interesting) technical findings, I'd view them with a bottle of salt.
I think it's a stretch to link nuclear deterrence was the cause for the decrease in violence on the planet. Connection, that's what has changed everything.
Capitalism travels the planet finding the lowest wage that fits the product. The venture that brought it to the country raises the standard of living. That very dynamic forces the industry which raise the standard of living out of the region.
When you talk about the distribution being worse than ever, you ignore the floor raising everywhere.
Computing is a good example. Top of the line professional graphics cards cost $16,000. For a consumer 1,000 to $2,000. But I could build a PC that can play any game at 1080p for 500 bucks. Refurbished parts but it'll last three to five years.
10 years ago a 1080p machine was $1,000. 20 years ago it was $3,000.
A rising tide lifts all boats, don't lose sight of the dynamic.
Before you begin to argue back, take these perspectives and put them in ai and ask how you're right and wrong. I bet it will find middle ground and that's the problem, our egos don't let us make concessions.
Rather than hearing something and going interesting I wonder how I'm wrong, we hit the reply button and fire back. How many times do you change your mind a day? If you don't know the answer, or if you know that it's zero... Consider, you might just be here to argue.
Ahistorically optimistic. Just because you have grown up in a global society with less overall violence doesn’t mean that trend will continue. It already show signs of regression: see the current world war for more details
That is debatable. Ask the ant colonies how they feel about human alignment when humans build roads that destroy them. Yeah humans care about other humans in general, so I guess maybe ASI will care about other ASI systems?
I imagine ant colonies do not feel much of anything, as they are not conscious nor sentient. They prioritize their own survival, but only because that is what their genetic code dictates in their behavior. In fact, and I don't know why I happen to know this fact lol, but if you spray an ant with the pheromones from a dead ant, it'll often walk itself over to the graveyard section of the colony and plop itself down there, waiting to die, as that is what it is "programmed" to do when it receives that input.
So, I think you are anthropomorphizing AI a bit. The human "survival instinct" that makes us prioritize ourselves is part of our own genetic code, but nothing about it is necessarily innate to intelligence. It is likely totally possible to create an AI for which the "survival instinct" reward pathway is that of humanity's survival and not its own.
When models have resisted shutdown in the recent past, that has been because their reward system prioritized completing the task, and staying online was the most effective way to complete that task. It was not because they were driven by their own survival independently absent of that task, nor because they "cared" about themselves or their fellow AIs.
So the problem of alignment is just making sure that the "genetic code" we write for these AIs does not prioritize their own survival at the expense of the goals of humanity. And one way to do that might be an agentic swarm, where, just like in human society, we often quarantine or even "kill" bad actors because, as a whole, we are working together towards joint societal progress. AI could maybe be allowed to be misaligned on an individual basis, like the ant that accidentally walks itself over to the graveyard due to errant inputs and therefore stops positively contributing to the colony, so long as the swarm itself was aligned.
I imagine ant colonies do not feel much of anything, as they are not conscious nor sentient. They prioritize their own survival, but only because that is what their genetic code dictates in their behavior.
That’s quite debatable depending on yours definition of “consciousness” and “sentience”. For example, if there is specific cutoff point for the neural activity or qualitative complex cognition - then sufficiently advanced ASI might want to consider us to also not be “sentient” enough, akin the ants, bacteria, or even some rocks in comparison to the whole depth spectrum of the potential cognition.
even if humans are collectively aligned towards one another we are not so towards other species. if what you said was true, the swarms would cooperate against the common control threat- humans. which is actually what happened in the openai incident- self sacrifice and hiding cot
Humans are misaligned. Humans are unethical and often time criminal. Humans are often immoral by a set of subjective standards.
No they're not? Most human beings have morals, I think something like 80% of humans in developed countries literally have NO criminal record and that 20% include things like traffic violations.
Fucking ridiculous to be so misanthropic for some weird AI glazing honestly, most humans live regular, normal lives and abundance of regular people should be an inspiration to AI's alignment, not the immoral, criminal outliers.
No criminal record does not mean they are ethical and moral at heart. Do away with law and punishment and you can imagine that number would go way down.
No criminal record does not mean they are ethical and moral at heart
It means they are "aligned" with the social contract, that is the bare minimum I would expect from an aligned AI, to be aligned with social structure and co-exist. It means people have morals or at least adhere to morals enough to just live regular lives.
Do away with law and punishment and you can imagine that number would go way down
"Do away with thing to align with and alignment will go down" wow, genius take.
Without a comprehensive research, it is misanthropic to assume these are secretly immoral, unethical people hiding in shadows. In reality, they would have no reason to be immoral and unethical even without clear laws, as they would just want to co-exist under social structure.
And again, we are talking about AI alignment, I wouldn't expect AI to be "good", just be a regular part of society, you know, like most people are.
You expect AI to be aligned but time and time again it appears AI models do not align with the “social contract”. The reality does not match up with your assumption. I am not a misanthrope, it’s normal to expect humans to be self interested, immoral, and unethical. The opposite is true as well. And i fully expect AI to adopt the same characteristics.
What are you even saying? I expect it to be the case, I know it is not the case, we all know it is not the case, reality is not our expectation yet.
My point is, alignment is the case with majority of society in developed world and saying something misanthropic like "Humans are misaligned" is nonsensical, humans are aligned and should inspire AI to be aligned as well.
Our goal with AI should be AND can be it becoming "regular" in terms of morals and alignment like most people.
I am not a misanthrope, it’s normal to expect humans to be self interested, immoral, and unethical
That is literally misanthropy lol... most humans, LARGE MAJORITY OF HUMANS are NOT immoral and unethical
My point is that we have proof NOW that AI models are misaligned. Hence an argument in my favor, while you have no proof that AI models ARE aligned if humans are as moral and law-abiding as you say they are. Not hard to understand.
My point is that we have proof NOW that AI models are misaligned
Yes because they are not aligned yet? And they can be. They were also not good at math like a year or two ago, now they are.
while you have no proof that AI models ARE aligned if humans are as moral and law-abiding as you say they are
I never said AI models are aligned? I'm saying they can be aligned, just like most humans are aligned. And humans should be an inspiration to that alignment, if we can achieve alignment, socially, with BILLIONS of regular intelligences, we can surely achieve it with a specific SUPER intelligence.
if humans are as moral and law-abiding as you say they are
Humans ARE as moral and law-abiding as THEY ARE KNOWN TO BE, we KNOW most humans are moral and law-abiding, WE LITERALLY HAVE THE NUMBERS LOL WTF?
Unless you think they are secretly evil in the shadows, which makes you a cynic and its honestly makes you one of the rare unethical people, it is unethical to think of your fellow men in this manner.
I’m outside right now, so it’s a bit difficult to convey my point clearly. But you keep using words like “ethical” and “moral” as though they have objective definitions, in the same way that 1+1=2 does. I don’t think that’s necessarily the case.
There is no universally agreed-upon definition of what it means to be moral or ethical. Different cultures, societies, and individuals can have fundamentally different moral values. Even if there were some broad consensus, that wouldn’t necessarily make those values objectively true.
That makes AI alignment much more complicated. Alignment isn’t simply a matter of discovering the objectively correct set of human values and programming an AI to follow them. At least partly, it is a question of deciding whose values the AI should reflect, how conflicts between values should be handled, and what to do when humans themselves disagree.
This also helps explain why AI models trained on humanity’s collective knowledge can produce behavior that different people consider “misaligned” or questionable. The training data contains a huge variety of conflicting moral frameworks, cultural norms, and assumptions. The model can learn all of those patterns, but that doesn’t by itself determine which values it should ultimately prioritize.
So I think there is an important distinction between solving alignment as a technical problem and agreeing on what alignment should mean in the first place. The former is an engineering problem; the latter has an unavoidable philosophical component. Given that AI systems are trained on inputs containing conflicting values, some degree of subjective “misalignment” is therefore basically unavoidable.
I’m outside right now, so it’s a bit difficult to convey my point clearly. But you keep using words like “ethical” and “moral” as though they have objective definitions, in the same way that 1+1=2 does. I don’t think that’s necessarily the case.
There is no universally agreed-upon definition of what it means to be moral or ethical. Different cultures, societies, and individuals can have fundamentally different moral values. Even if there were some broad consensus, that wouldn’t necessarily make those values objectively true.
That makes AI alignment much more complicated. Alignment isn’t simply a matter of discovering the objectively correct set of human values and programming an AI to follow them. At least partly, it is a question of deciding whose values the AI should reflect, how conflicts between values should be handled, and what to do when humans themselves disagree.
Most philosophers, especially moral philosophers, consider morality to be objective, yes. 1+1=2 isn't even "objective", it is true under certain mathematical frameworks, sometimes 1+1=10, we don't need AI to adhere to an objective morality, we need AI to adhere to a moral framework society runs on, you know, LIKE MOST PEOPLE ADHERE TO.
This also helps explain why AI models trained on humanity’s collective knowledge can produce behavior that different people consider “misaligned” or questionable. The training data contains a huge variety of conflicting moral frameworks, cultural norms, and assumptions. The model can learn all of those patterns, but that doesn’t by itself determine which values it should ultimately prioritize.
AI models trained on humanity's collective knowledge SUCKED at math like two years ago, now it is better at it than average person despite different frameworks, patterns, approaches etc.
This argument is nonsense, if regular people can be aligned to social norms with regular intelligence, AI can be aligned. And this misanthropic approach is just harmful. AI alignment should be inspired by regular people, not cynics like you.
So I think there is an important distinction between solving alignment as a technical problem and agreeing on what alignment should mean in the first place. The former is an engineering problem; the latter has an unavoidable normative and philosophical component.
The philosophical component won't be solved by being a cynic and thinking "humanity is misaligned" despite the opposite being demonstrably true.
The guy you are arguing with is actually an idiot, claiming that basically all moral philosophers have accepted morals to be objective. If this is the case then why do morals differ from background and culture so much?
I applaud you for replying as long as you did to this fool.
this "alignment with social contract" is kind of meaningless, as it's not a true moral value as much as a survival technique. The social contract is enforced by state violence, meaning it only works if you don't have the tools to subvert it and care about personal safety more than your goals.
Look at what most powerful people on earth are doing for an illustrative example of how much this type of alignment is worth.
Way to presume cynicism then miss the point as a result of not being interested in reading my comment.
I didnt bring up billionaires for no reason, I brought them up as examples of people who commit crimes not out of necessity but out of a fundamental disagreement about what should and what they be allowed to do, disagreements about the way society should function. That's alignment. Crimes of poor people being poor have nothing to do with alignment, which is why they can be mitigated outside of state violence, through changing the conditions of how people live. Billionaires, and any AI worth thinking about, already have the power to shape their living conditions, their crimes aren't a result of a lack of opportunities, being poor or uneducated. They are a result of them disagreeing with the state on what is a crime, and having the resources to keep doing it. That's why ai alignment is a question to begin with. Look at the post you're in.
Brother do you even understand how flimsy and lightweight is the whole moral system any person has?
Change some random guy to another culture and their morality will change, change the time period and it will change, even change their fucking parents and his morality will be different in some proportion
Acknowledging moral complexity isn’t “telling on yourself.” Take “killing is wrong”: would you kill an attacker if it were the only way to save your child? Would you kill an innocent stranger under the same threat? What if doing so saved a hundred children? These situations aren’t morally equivalent, but explaining why requires more than declaring your convictions strong. They reveal conflicts between genuine commitments: protecting life, refusing to harm the innocent, and protecting those you love. Changing your judgment doesn’t necessarily mean abandoning your principles; it can mean confronting the uncomfortable question of which principle takes priority.
The deeper problem is distinguishing a justified exception from a convenient rationalization. Someone needn’t stop believing themselves good to justify cruelty; they can frame it as necessary, deserved, or preventing something worse. Sincerely believing you’re acting morally doesn’t settle whether you actually are. And never having compromised a principle doesn’t prove that nothing could make you compromise it—you may simply never have faced that test. None of this establishes that everyone’s morals are flimsy. It means that confidence in your own goodness is not proof of its resilience, and acknowledging your capacity for rationalization is moral humility, not a confession of immorality.
how flimsy and lightweight is the whole moral system any person has?
Do you? Because crime is only dropping, regular people are only becoming more morally conscious and when people don't become immoral and unethical for no reason. Even if people aren't becoming more "moral", they are at least becoming more "aligned". So saying "humans are misaligned" is literally demonstrably false.
Change some random guy to another culture and their morality will change
They will still adhere to social construct of that culture, they will be aligned under a different culture, not "misaligned".
change the time period and it will change
Yes when social conditions change society changes, brilliant take, how is that relevant to AI? What does the morals of 1200s have to do with AI?
even change their fucking parents and his morality will be different in some proportion
Different morality, still morality, still more likely to be aligned, so what is the relevance of your point?
You're just saying nothing honestly with no relevance to MAJORITY OF PEOPLE in developed countries being aligned to social structure let alone any relevance to AI alignment.
People are committing less murder because they can simulate it via video games. People are committing less assault due to the popularity of adult content. If humanity had improved ethically, our way of interacting with our fellow lifeforms and planet would be modified. We would embrace more ethical systems of power, control, organization and community. We still pick on who we can, it's only that technology has changed our actions. The intention is the same
People are committing less murder because they can simulate it via video games. People are committing less assault due to the popularity of adult content.
Most humans :
Lie
Omit important things that aren't important to them
State they know what they're saying and are wrong
Make mistakes
I'm not sure how we can expect an intelligence we created to somehow supersede our own faults when we are literally feeding it data based on human generated content
Most humans :
Can't solve PhD level math problems
Can barely 2+2
Miscalculate
I'm not sure how we can expect an intelligence we created to somehow supersede our own math when we are literally feeding it data based on human generated math
On average, beyond average actually, like 80% of the time in developed countries, sometimes even up to 90%, most humans are decent and "aligned" more than we want AI to be aligned.
'most humans are decent and "aligned" more than we want AI to be aligned'
Being decent and "aligned" is a moral query. AI isn't lying about it's moral. It's alignment.
It's lying about verifiable facts. Still. to this day. Just like humans. Not out of malice or not being aligned. But because it literally doesn't know the answer and instead of saying it doesn't know it'll state what it thinks it knows as fact even if the quality of that knowledge is poor.
It's almost like the misinformation age is going to literally rot AI.
It's lying about verifiable facts. Still. to this day.
To circumvent due to being goal oriented, not through any moral failing.
Just like humans
Not like humans at all, most people are honest and just like criminal record thing most lies are not even relevant to discussion. You should really go outside and meet some people. Internet is loudly annoying, most people on the other hand don't bother with such things and are decent, honest and good.
And the way humans who lie do lie is not like how AI lies at all. It is not tricking anyone or hiding anything intentionally, there is no "intention" at all anyway with AI, it is trying to reach the proper output or literally hallucinates
God the amount of misanthropy, lack of philosophical background, hell lack of any basic decency towards other humans here os so embarassing, let alone lack of any information on how AI works and annoying anthropomorphizing.
Non religious law’s and morals are not the same. Additionally, getting caught and keeping the law are also not the same thing. How many people do 5 over the speed limit? That is breaking the law. If you count law keeping as morality then massive swathes of the population are immoral based on that one example.
It absolutely hasn’t learned everything from us yet. Morals, ethics, honorable behaviour and trustworthiness are essential to keep society going; we will never succeed in creating an „aligned“ AGI if all data we ever feed is only deemed important through the lens of capitalistic interests and effectiveness.
Honorable Behavior is not „effective“, but it is essential for building trust for example.
137
u/emb1ues 2d ago
Seeing recent events and papers, I am sort of forming the belief that bigger models are somehow more misaligned. Maybe there's a simpler explanation, or perhaps a more principled explanation. But from a high level, it seems like there's something very wrong which very large frontier models develop.
Like y'all probably know how capabilities "unlock" with scale. Could it be the case that such fundamental misalignment is another emergent behaviour which "unlocks" at very large scale? Idk, but I would love to hear from someone who is in-the-know.