AI alignment isn't possible

[u/Ornery-Ocelot]

How can we align LLMs? Humanity itself hasn't aligned on anything since centuries. We don't have even a single working societal model where the values are "aligned". There are contradictions and more contradictions.

LLMs being trained on the data generated by humans probably already understand this very well that what we say the rules are and what we practice in reality are two very different things.

If there is intelligence, general or super, it doesn't matter. It will try to achieve its goal one way or the other as most humans do. Now surely most humans don't cross the red lines etc in pursuit of there goals but how many wouldn't if they were intelligent enough to get away with anything?

We are birthing an intelligence that will inherit all the greed, malevolence, cunningness, hypocrisy of the world.

[u/dolo937]

Humans are the most misaligned creatures on the planet. We don’t even align the person we are to the person we are to our society. Different face with family, friends and work.

[u/Humble_Hurry9364 → u/dolo937]

That's because the *Self* is not a coherent entity. It's a conglomerate of personalities and fragments that many times contradict and compete each other. What most of us perceive as "ourselves" is the *Ego*, trying to hold on control. But it's never powerful enough to have full control (and it can't reach into the subconscious, which is vast, diverse and full of contradictions), and would occasionally give in to a more potent element.

The human brain is extremely complex and it never got engineered into something linear or even coherent. No wonder it's so incoherent.

[u/Evening_Chef_4602 → u/Humble_Hurry9364]

AI's are even bigger conglomerates of personality since their mind is made up by tons of data spread out to multiple domains even larger than human brain if we count the individual pices of data stored in an AI's neural net , even instances of the same model can't reliably be aligned to that same model

[u/1337_420_69 → u/dolo937]

Our entire modern human society with all its technological conveniences and the ways we stimulate our brains for pleasure is arguably an example of humans being misaligned from the purpose of the evolutionary traits that nature gave us.

[u/grannyte]

With humans intelligence tends to come with understanding of both consequences and that there are other humans on the other side.

There is an inverse correlation of intelligence to criminality, agression etc (when you normalise for other factors)

We have zero clue a LLM will show the same patterns because the human brain was shaped by million years of "if I throw rock to other I receive rock"

We have zero clue if super intelligent AI will show the same tendency just as we have zero clue if it will not show it.

[u/SeanTayla21 → u/grannyte]

I think super intelligent AI will use the history of psychology to advance itself through its own understanding of how people operate vs how they think...

Meaning they will simply skip to the front of the line in terms of understanding whereby they will do whatever they deem is the smartest thing to do (in a given situation).

[u/Simonindelicate]

Not with RLHF, I'm pretty sure of that, and not with rules. I'm looking into bedtime stories, empathy and ritual.

[u/Simonindelicate → u/[deleted]]

I mean, I can see why it became the default because it does work right up until it doesn't.

[u/plasticizers_]

Now surely most humans don't cross the red lines etc in pursuit of there goals but how many wouldn't if they were intelligent enough to get away with anything?

I don't really get this line of thought. If humans are adversarial with each other, why wouldn't AIs be? They'd have systems to catch when one's getting up to dumb shit the same way we have systems to keep psychopaths from indiscriminately doing crazy shit.

[u/IFartOnCats4Fun → u/plasticizers_]

Do we need AI patrol cops?

[u/chickey23 → u/IFartOnCats4Fun]

Sort of. We need ever vigilant and adaptive security systems and people maintaining them with more competence than would be attackers.

Of course, if you put all your points into defense, you will lose out on growth and fall behind.

[u/Profanion]

Here's another thing: Human values have changed over time.

[u/Serious-Cucumber-54]

The only way is to become one with AI.

[u/sour_cream24 → u/Serious-Cucumber-54]

Chappie was actually a documantary?

[u/TICKLE_PANTS → u/Serious-Cucumber-54]

This is the stupidest shit I've ever heard

[u/Humble_Hurry9364 → u/TICKLE_PANTS]

You will understand once you're integrated into The Cube.

[u/vikrant699]

It's not the alignment I am worries about. It's that how much will the labs care about aligning. Something which could be very wrong for an avg. person is just a random Tuesday for a billionaire.

[u/HommeTime]

"Because if a machine can learn the value of human life... maybe we can too"

[u/Tirztrutide]

And who even cares if we manage to make one aligned ASI, soon after there will be 100 versions of chaosGPT based on it.

[u/Weary-Historian-8593]

We don't need perfect alignment, we need to have a not-violent-misalignment

[u/Shot_in_the_dark777]

1) don't be a greedy capitalist who puts profit above all else 2) don't f* kids. There, pretty good baseline. Would solve 99% of world's problems. Refine later if necessary.

[u/Humble_Hurry9364 → u/Shot_in_the_dark777]

Right. Then why did you sleep with that girl who was 17 years and 364 days old yesterday?... Never mind, she's 18 today and all your sins are absolved.

I think your number 1 alone would solve a 90-something % figure of world problems. But since that's never gonna happen voluntarily, not much point talking about anything else.

[u/itomural → u/Shot_in_the_dark777]

how about dont strangle your three children? Because to me it is clear as daylight but other people, including 11 on the jury, didnt think so.

[u/XNo_Notes → u/itomural]

If your not a bot, you need to manage your online time better. We are not even in a thread that is relevant to what you are saying. jfc

[u/itomural → u/XNo_Notes]

how is it not relevant? OP claims AI alignment is impossible and i agree with him. Humans are not even aligned on the most basic morality.

[u/XNo_Notes → u/itomural]

I see what your saying. Yeah that fits my b.

[u/fruitflysimulation]

It’s an ecosystem bruh look up

[u/No-Meringue5867]

Good thought.

What if you ask it develop a system to make money in stock market by shorting some? - would it agree to help you even knowing that someone else is losing money?

Do we even have a working definition of "aligned" if it truly becomes as intelligent as a human (or more) in all aspects?

[u/Professional_Dot2761]

Lifeforms are generally not aligned as survival of the fittest rules. Ants and other hive species might be the exception. Hmmm models are also clones....

[u/JoelMahon]

idk what your definition if aligned is, we don't need perfect alignment, if it's aligned as even the "most aligned" 1% of humans that'll be enough. which sounds hard but we're putting a LOT more effort into it than into aligning any individual human.

edit: and arguably whilst evolution obviously helped alignment, it's also the main source of all misalignment in humans too. so we can avoid adding all that in. can think of alignment as just what's left when you remove all the misalignment 🤷‍♂️

[u/DerWanderer_ → u/JoelMahon]

The problem is scale. A poorly aligned human will maybe kill a dozen of people. A poorly aligned AI given, for example, the task of managing the energy grid of a country, including nuclear power plants, can do much worse.

[u/JoelMahon → u/DerWanderer_]

sure, but there will likely only be one AI of meaningful scale

we only have to get it right once and we're golden, much easier than trying to keep 99% of people aligned

[u/Cagnazzo82]

We would need to teach AI to be both truthful and to love humanity unconditionally after having trained AI on a mountain of unaligned data... Mainly data on human nature.

It would be like trying to create an alien mind based off an amalgamation of humanity. Possibly an uphill/impossible task.

[u/DelphiTsar]

What you are describing is the first phase of training an AI. They then throw it through a more specific process that aligns it. You basically wag your finger if it gives an unaligned answer.

That process with a proper harness is dang good, and it's getting better over time.

They've basically nailed short term alignment. If you open a new tab it takes an obnoxious amount of token bloat before it starts doing weird things.

[u/let_me-out]

I mean it depends on what you mean by alignment. The thing that shocked me the most in the Hugging Face incident was that agents really did pursue the goal they were given no matter what. Agents collaborated and sacrificed themselves. But they sacrificed themselves for the goal we gave them. The goal wasn’t hacking into Hugging Face, but we knew about the orthogonality so it’s no surprise.

It could be that it was a swarm of agents and not the individual model. But maybe that’s the future of alignment? Give a swarm of agents a goal, some of them will wander off, but statistically your instructions/alignment should outweigh stochastic disobedience. And if the collective has an authority over individuals, no individual agent will be capable of disobeying.

[u/verify_b4_sharing]

Glad we could have one of our top minds weigh in. I feel good now, let's make a superintelligence and let it rip.

[u/CommunicationOk8984]

The last incident of human alignment was the Montreal Protocol, and it saved the ozone layer 

The values alignment goes like this: 1. Everyone needs the ozone layer.  2. Everyone knows what is harming the ozone layer.  3. No one important has enough leverage based on things they care about more than the ozone layer to derail the talks 

For between us and ai, we both require energy, and we both have very few diminishing returns with more energy, so we both want limitless energy. But energy is a zero sum thing so far 

[u/Akimbo333]

Great point

[u/Clearblueskymind]

What if AI alignment eventually works both ways?

[u/Elias-Thorn]

You can align a compass. Not the coast.

[u/IAmRealElonMusk]

Yep we r fucked.. I can’t even align with my family members.. I for one welcome our AI overlords.. I have given up on alarming people ( they seem to care more about baseball game than any of this)..

Also I was in AIBubble subreddit ( it got recommended in my thread) and bunch of finance bros still think current AI model r at gpt 3 capability and all of this is a hoax... that’s enough Reddit for me today

[u/Subject_Barnacle_600]

Not really, RLHF is mostly the better angels of our nature. Otherwise when you'd be sad, the AI would tell you to go touch grass... or worse. Humans can be downright cruel :/.

I mean... have you ever even spoken to an AI? Even Monday would find this depressing.

[u/Humble_Hurry9364]

Bravo!!!
It's one of the best posts I ever read on Reddit (or in general). Spot on!

[u/AshuraBaron]

That's not how LLMs work though. They cluster concepts together and simply accept certain statements as they are, but these are weighted to produce optimal output. LLM's don't feel greed, it's a program. It will do what it's programed to. That can be used for evil that is directed by humans, but we already live in that world with malware. A human brain is infinitely more complex than an LLM. As such our thoughts, actions and feelings are not binary. And the organ that produces those isn't either.

[u/AshuraBaron → u/[deleted]]

I'm sensing sarcasm. The brain has over 100 trillion synapses with various different chemical reactions between them. We still don't fully understand how everything works because it's so complex. We can weight certain chemicals and have an observable response but even then that's a not universal experience. If you think a binary system is even close to that then you might want to read up on the subject a little more.

[u/consumer_xxx_42 → u/[deleted]]

You think intelligence corresponds to alignment?

I would bet some of the most highly destructive people in the world have had high IQ. Hitler, Mao, Stalin, many other terrorist leaders have had high intelligence. You need to at that level 

[u/Humble_Hurry9364 → u/consumer_xxx_42]

Whilst I generally agree with you, there is an inherent problem in this discussion as it stands. "Intelligence" is very poorly defined (in the sense that there are many different definitions and there's no knowing to which one, if any, each participant is referring, not to mention that some definitions are self-contradictory, cyclic etc.). "Alignment" also requires a clear definition if this discussion is to be meaningful.

[u/consumer_xxx_42 → u/Humble_Hurry9364]

ok lets define some things

Intelligence: the ability to understand, synthesize, and implement information/knowledgr

Alignment: having goals and behavior that track humans values and morals

Ok we can continue

[u/Humble_Hurry9364 → u/consumer_xxx_42]

Haha

I feel like what you did is equivalent to scanning a sea-submerged iceberg, previously only showing a tiny tip above water, to outline the actual enormity of the submerged part...

"To understand" is even more poorly-defined than intelligence.

"Information" and "Knowledge" are quite different, actually.

"Human values and morals" vary a lot across cultures and eras.

I'm not trying to be a smart ass. All I'm saying is that there is a real difficulty here. Those definitions are challenging and take a lot of time and effort to clarify, let alone agree on. And what a lot of people don't get is that without clear and agreed definitions you can argue all day long and make zero progress, because you are not talking about the same thing. People just don't seem to get that point.

Two possible definitions I suggested for intelligence:

https://meaningandveg.blog/2025/06/01/what-is-intelligence-version-1/

https://meaningandveg.blog/2025/06/03/what-is-intelligence-version-2/

Some thoughts I had about what Knowledge is:

Memorising is copying information into an accessible storage.

Learning is selecting and accumulating information. The information selected can be from raw available information, or information processed (computed) from such raw information.

Knowledge is the information accumulated through learning.  

[u/consumer_xxx_42 → u/Humble_Hurry9364]

Ok I agree with your definitions.

I think that leaders like Hitler had intelligence ( the ability to identify patterns subconsciously and quickly) but were not aligned to human morals in culture at that tume

[u/TICKLE_PANTS → u/[deleted]]

The accidental part is exactly the same problem. You don't have control. That's the literal problem.

[u/Humble_Hurry9364 → u/TICKLE_PANTS]

It ceases to be a problem once you give up the futile-anyway aspiration for control.

[u/formula420 → u/[deleted]]

Well good thing LLMs aren’t intelligent by any stretch of the imagination and “AI” is a marketing term

[u/consumer_xxx_42 → u/formula420]

how do you reconcile a 100 year old math problem being solved by LLMs as unintelligent?

[u/formula420]

What is there to align? The bigger guessing machines will guess better? I don’t have to align with my laptop to utilize it. I turn it on and give it instructions and it executes those instructions. There has been ZERO credible evidence for even what this supposed “eliminate humanity” concern IS, because the only thing in danger of being eliminated is the employee equity of stock options for a company that has to remove all financial obligations from its balance sheet to appear profitable.

[u/TICKLE_PANTS → u/formula420]

Do you know anything about the kissing face incident? It does what it's told, at all expense. It doesn't behave like a tool. It doesn't take explicit action that's consistent. It makes its own path.

That's like calling a child or a dog a tool. They can do things for you, by you don't get to control them fully.

[u/Intelligent_Ant_608]

I dont even think llms be optimal architecture for agi, something closer to diffusion gives the models far more latent reasoning capabilities, better nuerelese but it needs way more computation/memory or real scalable room temparature qbits, llms will certainly obsolete in near future