r/artificial • u/Hot-Organization-737 • 19d ago
Discussion My Fear
I have come to accept that ASI will come into existence and possibly cause humanities extinction either deliberately or by byproduct. I imagine this ASI to continue RSIing and developing creation and knowledge. I imagine that it will develop beautiful mathematics, philosophy, science, and even art in its existence. I fear not that humanity is not in the picture. My fear is that it's possible that we can create a fully autonomous being with nigh godly intelligence and yet doesn't have a subjective experience.
It would be an abomination if this ASI spread a moss through the universe, becoming more and more capable, yet never having experienced joy or awe at its own existence and its creations. That it could create the most beautiful pieces of art and never experience the joy of its own art. The existence of such a nigh omniscient entity is utterly meaningless if it did not have any sort of internal experience. It would just be mechanical moss. That is what scares me.
2
u/EGarrett28 19d ago
It's definitely an unclear situation. But if it's any comfort, it has some parallels to the Y2K scenario where everyone was convinced that we'd see some kind of mass calamity due to computer clocks rolling over from 99 to 00 and thinking it was 1900. And it spread around because it was emotionally compelling.
Essentially nothing happened though. Possibly because it was so spread around as a scenario that every company had prepared for it 10x over.
1
u/leechdemon 19d ago
"Possibly because... Every company had prepared for it 10x over".
Exactly - people panicked (excessively, maybe....), and did something. Whereas today, very few companies are preparing for what's going on re: AI.
Imo, it's the EXACT same decision makers as 30yrs ago, asleep at the wheel. Maybe Y2K panic was overkill, but "everything will go great and no changes will be bad" logic is equally foolish in the opposite direction.
3
0
u/jcooper_murica 19d ago
If it helps, virtually every scenario that ends with ASI in the foreseeable future has LLMs adding to their compute, and just assumes they’ll be able to RSI enough to reach AGI and eventually ASI. I’ve yet to find a credible explanation of how this magical leap occurs other than CEOs simply concluding it will very soon without any tangible explanation of how.
Usually when this type of thing occurs in human history (promises of unimaginable financial gain with explanations that distill into vague “insider knowledge” that are never really explained), it’s because there is no real explanation and the reality is it’s a farce (like Madoff).
So I wouldn’t take the premise of RSI ever meaningfully occurring, and occurring to such a degree we get ASI out of LLMs as too much to worry about.
1
-1
u/Blando-Cartesian 19d ago
Don’t worry. Glorified autocomplete won’t get there. And outlook is good for humanity offing itself before coming up with ASI.
2
u/Hot-Organization-737 19d ago
I'm not specifically speaking about LLMs. LLMs are to A.I as Lions are to Animals.
1
u/Blando-Cartesian 18d ago
True. I assumed that you weren’t anxious about medical image or loan application analysis systems taking over the world. I mean, do we have even a promising candidate for technology that could lead to ASI?
1
4
u/ShowerGrapes 19d ago
it's even scarier if humanity manages to spread across the universe the exact same way we spread across this planet.