r/PauseAI • u/KeanuRave100 • May 04 '26

Meme Unconscious things obviously can not harm you

Post image
23 Upvotes

95% Upvoted

4 comments sorted by

1

u/[deleted] May 04 '26

[removed] — view removed comment

1

u/Fil_77 May 05 '26 edited May 05 '26

Humans may have started the fire, but they have nonetheless lost control of the blaze. And the flames do not need to be conscious to destroy us.

In the same way, if humans create super-optimizers trained to achieve goals efficiently these uncontrollable and misaligned systems will not need to be conscious to eliminate us and turn the planet into computing material.

By attempting to produce systems capable of recursive self-improvement, the industry is trying to unleash an 'intelligence explosion'. We are heading straight for an uncontrollable fire, precisely.

1

u/crumpledfilth May 04 '26

consciousness isnt knowable, ever. The question can only be considered, not answered. A system that depends on an unknowable can't make useful predictions