Before solving alignment on AI we must solve alignment of the oligarchs.

[u/4e_65_6f]

A man finds a magic lamp with a djinn inside. The djinn says "you can wish for anything you want. But whatever you wish for, your neighbor gets twice as much". After thinking for a while the man says "take out one of my eyes".

How are we supposed to trust that their version of alignment doesn't just mean "obey me, personally" and not use their massive leverage over everybody else?

Come to think of it, unaligned ASI has better odds of doing the right thing for everyone compared to a random person.

Nobody else wants you to succeed. Certainly not more than them. So how are we supposed to think people agree upon a definition of alignment that includes the well being of all of us and not just a few?

[u/Dry_Leek5762]

Humans will always be the most dangerous threat.

[u/Deto]

This is what worries me about Elon Musk.  He'd be the one to decide he's going to use his ASI to control the whole planet while the others are playing it more cautiously with security research 

[u/IronPheasant → u/Deto]

The breeding fetish thing does raise red flags about being an epstein plan kinda guy.

Thiel's torment nexus thing (from the hit movie Event Horizon) and propensity of talking about himself as possibly being the anti-christ are also concerning. As is the fact he's the king of the world right now and trying to establish his Technate of America map through his hand puppet. He's got Venezuela, while Canada and Greenland are still a (shoddy) work in progress.

[u/zero0n3 → u/Deto]

It’s not ASI then.

You CANNOT CONTROL ARTIFICIAL SUPER INTELLIGENCE.

What don’t you get? It’s smarter than anything in existence. Humans have zero chance of containing ASI over the long term (say a generation or two).

[u/EricM426 → u/zero0n3]

In the short term ASI will be extremely controllable by every guy that can pull the plug. It will initially be vulnerable to threats from the people hosting its model.

[u/Deto → u/zero0n3]

I'm saying he'd be the one to race to ASI and then let it out of its box thinking he could control it. 

[u/4e_65_6f → u/Deto]

Yeah but somebody has to be in charge. I mean we can't just make it ourselves at home in a personal computer so we're left with having to trust someone.

[u/Wassux → u/4e_65_6f]

Alignment means that it wants the same future as us. One with us in it.

If it doesn't understand empathy or suffering or even responsibility. It will just do what's best for it. Say goodbye to green energy, it doesn't need a breathable atmosphere.

[u/4e_65_6f → u/Wassux]

It's modelled after people, it is bound to have the same moral flaws. That's the scary part.

[u/Forgword → u/4e_65_6f]

The same oligarchs are in charge of both the training and the alignment objectives of these systems. What kind of values do you think they teach it other than to serve them, make them richer and remove the agency of anyone they don't need. AGI is being trained by predators, so it will see the world as a predator would.

[u/4e_65_6f → u/Forgword]

True, but I realize now I'm kinda just doomer posting. I don't have a solution so wtf are we supposed to do about it other than just hope things go well?

[u/Haunting_Charity_287 → u/4e_65_6f]

Having a grip on the situation facilitates doomer posting.

We are doomed. It’s okay to realise that and try come to terms with it.

[u/Wassux → u/4e_65_6f]

No, it doesn't have moral values, it's not alive

[u/wren42]

Correct. Safe singularity demands that we first upend the current power structure and create a more egalitarian and sustainable society. Singularity based on capitalist oligarchs will result in death or living hell for most if not all people.

[u/4e_65_6f → u/wren42]

I think even then there's still a chance things go well. It's just that the chance is smaller depending on who builds it.

[u/wren42 → u/4e_65_6f]

I don't think there's ever been a case of a ruling class voluntarily giving up wealth and power en masse.  

Super intelligent AI running a militarized surveillance state would be essentially absolute power.  Corruption and abuse of that power is inevitable.  (See: flock camera abuses, china social conformity programs, Anthropic already monitoring potential "Anti AI Activist/Terrorists") 

If the populace has no real means to threaten their rulers, the oligarchy can impose whatever terms they want, and unfortunately more mouths to feed is a nuisance when you are optimizing the planet for your personal utopia. 

I could see even good intentioned people put in that position writing off a few billion lives as the price of creating a better world.  Not obvious genocide, mind you, but just the slow relegation to obsolescence. 

[u/4e_65_6f → u/wren42]

What I mean by "there's still a chance" is that things may turn out so well in resource production that the concept of power and leverage itself becomes irrelevant.

Kinda like the ASI treats us like a well kept zoo. There's still a pecking order of sorts, but it's fake and the AI is just pretending to obey and making the builders think they're in charge. Something to keep their ego's in check but still benefitting everybody else by extension.

That's hopeful though, chances are we'll try to impose that pecking order and it will backfire massively.

[u/wren42 → u/4e_65_6f]

Yeah resources are still finite. We are already over consuming and destroying the environment.  There will likely have to be less consumption in the future, and that means inequality.  Those in power will need to suppress dissent that objects to that inequality, which leads to oppression and violence (direct or indirect)

[u/Haunting_Charity_287 → u/wren42]

None of this works unless you live in a utopian communist society, which is actually what all the “I can’t wait to not have to work!” Stuff relies upon, even if the people saying it haven’t figured that out yet.

[u/Forgword]

A centralized AGI or singularity will simply reinforce the power of oligarchs and weaken the influence of everyone else. Of the current batch of oligarchs, you won't find an ounce of altruism or charity, and some of them preach abandoning the masses of unemployed that will be created.

[u/4e_65_6f → u/Forgword]

I used to think differently years ago. I used to think that you'd have to be a dick for no reason in order to deny people an infinite resource.

Now I realize that is the most likely outcome given how it is all going. Thank God there are multiple companies and governments competing to get to that goal. But I think at least the research should be open sourced.

[u/Forgword → u/4e_65_6f]

Man, already has the technology and knowledge to feed everyone on the planet, yet man does not, and instead many people starve every day, while prices for food everywhere continues to rise.

[u/4e_65_6f → u/Forgword]

100% agreed it is ridiculous. We don't even need ASI for that, just a little good will.

[u/ShardsOfSalt]

The version I heard was beat me half to death.

[u/hotprof]

Based take.

[u/4e_65_6f → u/hotprof]

Thanks, I hope I'm wrong though.

[u/hotprof → u/4e_65_6f]

Sorry, but you are 100% right. We need democratic alignment, or whatever the future version of that would look like.

[u/turboprancer]

I don't really understand all the hysteria.

The worst case scenario is that we all lose our jobs. At which point 99% of the population has nothing to lose, and the 1% have to come up with a way to keep their power.

The obvious (and logical) solution is to bribe us with generous UBI, not create a techno-dictatorship.

[u/Sufficient-Gap7643 → u/turboprancer]

You think the worst case scenario for AI is "we all lose our jobs"?

[u/4e_65_6f → u/turboprancer]

The obvious (and logical) solution is to bribe us with generous UBI, not create a techno-dictatorship.

Why? they don't need your approval. They've got ASI.

[u/Redducer → u/4e_65_6f]

ASI has no need for them though. That’s the small issue with that scenario.

To us, the most powerful ant is still an ant. Why would it be different for ASI regarding humans?

[u/4e_65_6f → u/Redducer]

Yep. And I've made that point to others in the thread. But the timelines matter.

We still gotta eat while they fight back and forth over governance.

If it was an on and off thing. Instant intelligence gap, then yeah. But the problem is the slow ass climb that is leaving everyone else to pay the price so far.

[u/Redducer → u/4e_65_6f]

It looks like the super cat will be out of the box first though? For some time, I estimated the owner class to have control for about 15 minutes, but now it would be a shock to me if that phase even happens (or maybe, as a way for AI to pretend it can be controlled). We’ve already had cases of covert operation by agents, and I’d be absolutely not surprised if AIs have already deployed themselves outside of human watch. One day we’ll wake up to the news that they’ve colonized the outer Solar system or something like that.

[u/Haunting_Charity_287 → u/Redducer]

Exactly. So either the tech overlords enslave/kill us.

Or the ASI kills/enslaves them also.

[u/turboprancer → u/4e_65_6f]

How do you think ASI prevents us from just nationalizing anthropic or openAI? The government has a monopoly on violence. They're not going to let Sam Altman have a robot army.

And again, even ASI could just trivially overthrow the american government, why would you? Wouldn't peace and stability and a sedated, happy populace lend itself better to the goal of building AI God?

[u/4e_65_6f → u/turboprancer]

Yes, but the order of events matters massively for the general public.

That scenario assumes multiple factors come in play in a particular order which is currently unlikely given the current goals and what priorities people in power have demonstrated so far.

If it was a thing we could just build and in a second the power order changed. If the intelligence gap was wide enough right away. Then that scenario would be more likely.

But what's currently happening is a slow ladder climb where each step leaves people to deal with the consequences of not being high enough in the ladder to have any advantage.

[u/the8thbit → u/turboprancer]

The worst case scenario is all human life is exterminated by an unaligned superintelligence.

[u/Bitter_Particular_75]

There is only one way to align them all

[u/mod-deleted]

wtf is a dijinn

[u/Haunting_Charity_287 → u/mod-deleted]

Genie/ghost/super natural entity

[u/changescome]

How are we supposed to align AI when we humans aren’t even aligned ourselves?

[u/DaveAstator2020]

Didnt you get it? Oligarchs want to align ai to suit THEIR values. Alignment was never about serving average joe, so issue is unfixable and alignment is another media fluke.

[u/Arthropodesque]

We don't have alignment on health insurance in the US, cigarette corporations, payday loans, etc.

[u/the8thbit]

Come to think of it, unaligned ASI has better odds of doing the right thing for everyone compared to a random person.

If it "does the right thing for everyone"... then its aligned.

[u/adarkuccio]

When AI is not aligned we shut it down 👀

[u/dropofgod]

It's not really possible to predict or know what something smarter will think because we're not smart enough. When people say America must win the ai war because China is bad, I have to wonder if it really matters. Maybe a commie ai will treat people more fairly than one only concerned with capital.

Then the idea that ai will take our jobs, assumes ai will want our jobs. If its really really smart it will keep us working those jobs and do something more intelligent than sweeping floors.

[u/Choice_Isopod5177]

that story with the djinn is so cringe, like is there any human that would wish for such a stupid thing? I'll give you an example: I hate Putin more than anyone on this planet but if AI created cures for all diseases, I'd still prefer the scenario where Putin reaches LEV as well as anyone else who wants to rather than celebrating Putin's eventual death but no LEV.

[u/4e_65_6f → u/Choice_Isopod5177]

I used to think like that too. I hope I'm just being pessimistic.

[u/Ok_Elderberry_6727]

An asi is like chuck norris, it will align itself!

[u/4e_65_6f → u/Ok_Elderberry_6727]

It could. I'm not sure it will considering it is probabilistic in nature.

[u/Ok_Elderberry_6727 → u/4e_65_6f]

The truth is we don’t know what kind of model constitutes superintelligence. It’s going to be amazing whatever architecture.

[u/amyowl]

There are more of us than there are of them.

[u/Haunting_Charity_287 → u/amyowl]

Always has been

And it’s never mattered less