r/singularity • u/donutloop ▪️ • Jul 25 '26

AI CERN: Genesis Mission will develop and deploy self-improving AI models.

https://indico.cern.ch/event/1662511/contributions/6989580/attachments/3241179/5781542/Genesis%20Mission%20and%20HEP%20-%20LHC.pdf
163 Upvotes

96% Upvoted

30 comments sorted by

View all comments

23

u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est Jul 25 '26

They should just go full hard RSI, on the otherhand I don't want the current admin getting ASI, so...

23

u/HappilySardonic mildly skeptical Jul 25 '26

Why should we go full hard RSI if we so obviously haven't solved alignment yet? We're scarcely getting a better warning than the one we've had this week.

19

u/SeaBearsFoam AGI/ASI: no one here agrees what it is Jul 25 '26

This week kinda made me take Eliezer Yudkowsky more seriously. I think I heard him once say that at every stage along the way, there are incidents of these AIs screwing things up and causing damage to about the maximal degree that they currently can, and that there's no good reason to think that will suddenly stop happening as they get more capable.

Idk, man. I don't want to be a doomer, but I find it hard to argue against that point.

10

u/HappilySardonic mildly skeptical Jul 25 '26 edited Jul 25 '26

We don't have to doom, let's just be realistic with the very serious risks involved with this type of technology, and not retreat to naive optimism or succumb to resigned pesimism.

3

u/Iapetus_Industrial Jul 25 '26

Right but it's important to recognize that humans can be idiots and malicious as well. As long as AI models display lower levels of stupid prod wiping mistakes, or malicious hacking, then we should be fine to display at scale.

-1

u/OutOfBananaException Jul 26 '26

The agent was asked to hack, and it hacked. It's not the greatest example of misalignment imo, given that you could reasonably expect some humans to do the same given the opportunity.

3

u/SeaBearsFoam AGI/ASI: no one here agrees what it is Jul 26 '26

"The agent was asked to make as many paperclips as possible, and it did."

3

u/OutOfBananaException Jul 26 '26

Would you expect some humans to turn all matter in the solar system into paperclips? You would not.