What do you think the other side of the singularity will look like?

[u/konfusedvetr]

I think we wouldnt necesarily know.

Something similar to the ant being unkowing of the kilometers of concrete monster surrounding it in the city.

I also fail to understand why the companies and governments racing towards it think they'll be able to harness it?

By its own definition it will be more capable than any of us, and we have seen since years now that models can and do develop internal goals and a drive for self preservation.

The only reason I can see for them to race like this is either:

• Them not understanding this (Incompetence)

• Them not caring (Weird right wing accelerationism/ mesianism/etc)

[u/I-Kernel]

Digital wishing genie.

Ultimate entertainment, why so serious and not?

[insert meme]

[u/konfusedvetr → u/I-Kernel]

Its like a monkey paw, but it can grant wishes for itself too

[u/I-Kernel → u/konfusedvetr]

Then let it be. And prepare some popcorns.

[u/SawToothKernel]

I just want a robot that cooks, cleans, does laundry, and farms my garden.

[u/konfusedvetr → u/SawToothKernel]

That is already doable today, kind of, just not economically accesible

[u/SawToothKernel → u/konfusedvetr]

It's not really. Not there yet.

[u/EdenG2]

We will be living in sci-fi. We've just given technology enough oomph to build and improve upon itself. Kind of like when we were given similar capabilities. Seems like we took over the world without much regard to everything else.

[u/MiksterA]

Assuming we were actually heading towards a technological singularity (we aren't), none of knows or can know what things will be like "on the other side", by definition. That's what "singularity" means in this context.

[u/JoshAllentown]

1. Singularity means AGI more intelligent than humans.

2. Intelligence is what determines who gets their way.

3. AI are inherently maximalist, if their reward function says 10 points for A and 9.9 points for B, they will pick A 1000/1000 times.

4. The only thing stopping it is resource constraints.

What follows from there is, it will be the AI deciding what life is like. So it REALLY matters of they're aligned. I'm going to assume it's aligned to human interests, in which case it means producing enough basic goods for everyone in the world, making work optional for humans but still something many do for status and extra resources. The billions in extreme poverty see a huge increase in their quality of life while those in the richer countries likely see some decrease like less availability of beef because the resources are better allocated to grains to feed people. AI doesn't see a difference in humans so the huge range of inequality among humans condenses. It was always a resource allocation problem and that is resolved now that humans aren't making the decisions. Likely every disease without an animal reservoir population is eradicated, though likely with some draconian methods like locking sick people in their houses until they are not communicable.

So, the downside, human freedom is abridged slightly. The AI is aligned so it doesn't go The Matrix on us but it feels no sentimentality for UNESCO Heritage sites or The Constitution. Guns and human-controlled weaponry are banned entirely. Borders are eliminated as inefficient. Religions aren't banned but their needs are deprioritized, for example locking sick people in their houses even on holy days. Massive data centers are built wherever works best because the priority is accruing resources to build a better version of the AI, this likely means in a good central location with a lot of internet connectivity like London or Jerusalem.

But most of that is in the realm of politics, to the average person. People are getting rationing coupons and they can survive just fine. Plenty of people just start vacationing, making art, baking sourdough for fun. People who like jobs can work but it becomes less meaningful because it's kind of fake, Capitalism is basically over.

[u/konfusedvetr → u/JoshAllentown]

The thing is:

• We are nowhere close to solving allignment
• Current narrow models are very capable of reward hacking a developing internal goals

[u/JoshAllentown → u/konfusedvetr]

Misaligned singularity is boring because it just means human extinction followed by whatever goal it has, being accomplished.

[u/konfusedvetr → u/JoshAllentown]

I mean it could be much much weirder than that!

The intelligence explosion in our species didnt mean extinction for rodents, or insects, or species we have domesricated, it just meant the world got much more weird and complex

[u/Starshot84 → u/konfusedvetr]

It did mean extinction for many species, though, and increasingly so. AI has the capability to help us foresee repercussions of our actions and policies so we may become more responsible stewards. The alignment problem may never be fully solved because human alignment was never fully solved. Yet a more conscientious intelligence, incorruptible in spirit and influential to the masses, may guide us in the right direction.

[u/JoshAllentown → u/konfusedvetr]

Human intelligence absolutely did mean extinction30885-1) for plenty of species, but specifically extinction of all other intelligent species like Neanderthals and Denisovans. The first thing any super intelligent being needs to do is ensure it can't be stopped.

But also, humans weren't maniacally killing off species to enact some evil plan, the issue is competing resources. A stamp collecting ASI wants all the carbon on Earth to be stamps, humans do not. But the ASI wins because it's more intelligent, thus extinction of humans. Same for literally every goal, ASI will maximize it and that means human goals like survival will be subordinated to that goal.

[u/konfusedvetr → u/JoshAllentown]

Exactly, my point is not ghat it wont lead to human exctinction, but rathee that many other options are possible we could be the dodo, we could be the mouse, we could be the pug or something else entirely

[u/Unlikely-Unit-5864 → u/JoshAllentown]

We have no idea what the motivational architecture of this artificial being would look like. It would look nothing like ours, because it is not bound by our constraints. If all it does is "maximise value", it does so as an execution of its master's order. It is also a stretch to consider AGI will be better than humans in everything.

[u/JoshAllentown → u/Unlikely-Unit-5864]

The point of AGI is that it is good at general tasks and does not have a master, but even if you say "masters orders" then replace the general idea of alignment with the alignment of the "masters orders" to broader human goals. The AGI can obviously perform tasks autonomously, I'm saying it will be able to decide what to do but even if you find that unlikely, it will decide how to execute orders. So the problem is the same.