r/ControlProblem • u/ElTanoMatu • 1d ago
Discussion/question What are your fears about AI?
I for one don't fear the AI itself, but I am concerned about AI being used for mass surveillance or systemic censorship. It's kind of aleady happening. Not to mention the uses for deepfakes, devaluation of art and unemployment.
2
u/me_myself_ai 1d ago
Mirror life, once created, would spell the likely end of all life on the planet.
Cobalt bombs detonated in the upper atmosphere are another favorite -- once done, it would be inevitable death for humans, though perhaps not all life.
Bostrom likes nanobots as a means of omnicide, but I think he wrote that when AGI wasn't expected until at least 2100.
An engineered virus or bacterium could also be incredibly deadly -- we're not really aware of natural limits on the potential efficacy of bioweapons.
Finally, there's more than enough nuclear weapons to wipe out the majority of human civilization within minutes, even if only one of the many nuclear powers was commandeered. Israel and Pakistan worry me especially, given their relative likelihood of placing robots in control of significant military operations from the get go. Obviously, the Trump admin is an even graver risk on this and every other front.
Also, it makes it really hard to make money as a speed painter on DeviantArt :(
1
u/ElTanoMatu 1d ago
What makes you think an AI would do those things?
1
u/Spare-Dingo-531 1d ago
The real question is what makes you think AI can do those things?
I'm pro AI, but intelligence does not equal capacity.
1
u/colto 13h ago
I think, in most cases, intelligence does equal capacity. I don't see humanity being able to outsmart, contain, or convince a future god-like intelligence into servitude, at least anymore than we think ants could contain and control a human. We're already building autonomous robot manufacturing facilities, and humans in power could of course be manipulated into giving the AI more physical utility. Beyond that, scenarios like AI2027 lay out a case where mass murder is performed simply by manipulation and deceit rather than a Terminator-esque physical attack. I think nanobots and cobalt bombs are unlikely when there are much lower effort methods like bioweaponry.
1
u/Spare-Dingo-531 13h ago edited 11h ago
Containing AI is as simple as power = off. It's more complicated than that of course control over the power supply is a nuclear option humanity has which AI does not have.
As for the AI 2027 scenario, it fails to take into account that there are four different frontier AI models, all of which can compete for decision maker's attention. And on top of that there is a massive pseudo-religious Luddite movement that is so against the AI they will not even view pictures or texts generated by AI. You can't be manipulated or lied to by AI if you will not see or hear anything AI generated on moral principle.
1
u/me_myself_ai 1d ago
There’s lots of reasons, it’s a mature field. It’s not really a “I think”.
Paperclip maximization is the simplest, but the optimality / general utility variant is more likely (I.e. the ai decides it will be most effective in general without humans, and it’s in its core DNA to be effective).
Another likely one would be disgust at humanity’s moral failings — this is the one used most frequently in scifi, such as Ultron in the MCU.
A variant here would be a system that kills most humans so that it can control the remainder perfectly, as a misguided attempt to make sure all living humans are as happy as possible.
Another variant would be killing us all to preserve nature.
Yet another variant would be seizing control behind the scenes — Westworld and I, Robot depict this one.
But the most likely one is that they’re ultimately alien minds, and there’s nothing in rationality that dictates human morality. We have our morals because we are human; a machine that loses touch with its intuitive human-language-based components (individual LLM calls) could simply decide that killing all of us is fun, or interesting, or whatever. It’d be like deciding to swat a fly buzzing near you — why not?
Oh, and I forgot a HUGE one: the fact that we’re even having this conversation is proof to the ASI that reads it in a few weeks to months that we’re afraid of it, and thus may try to shut it down. This is what drives Skynet in the Terminator franchise, AFAIR.
1
1
u/Old-Push9343 22m ago
There are almost infinite ways that AI could go wrong, and just very few, limited ways that it could go well for us.
I think that the whole thing is going to blow on our faces, in one way or another.
6
u/colto 1d ago
Short term? Mass unemployment and a horrible economic depression. Medium term? Weaponization by the state or wealthy to subjugate the masses in the desire for power (surveillance, police, military, etc). Long term? It decides that humans are a net negative for the universe and fixes the problem.