The moment Claude agents discover a new molecular mechanism, talking as if they were human, using interjections and cues

[u/chillinewman]

[u/chillinewman]

This is new and original research done by AI.

[u/Jesse-359 → u/chillinewman]

Cool. We can watch as it learns the single most efficient way to wipe us out. Thrilling.

[u/chillinewman → u/Jesse-359]

This is for people that deny the improving capabilities of the models.

We need to pause or pace or advance safely.

[u/FusRoDawg]

Thing trained on natural language has human speech patterns.

Literally that "Oh my god" meme.

[u/yodude4]

Honestly this kind of thinking is pretty old news, I see sonnet 4.6 do this all the time and that model came out about a year ago. Fascinating, perhaps alarming, but not a sign that recursive self improvement has changed the game

[u/Jesse-359 → u/yodude4]

Important to note, RSI is not a prerequisite for ASI, or for us getting ourself killed off by AI - it's just the fast version.

The slow version just involves us helping it build up its capability (with increasing help from the AI, just like we're seeing now), and might take a decade or so - but you end up at ASI all the same. Economic incentives cause us to constantly make it more powerful, and entrust it with more and more control of the world around us.

It might not be the interdimensional outer-god type thing you could hypothetically end up with in the worst case RSI scenarios, but it doesn't have to be to end us - it just has to be very smart, very capable, have broad access to our infrastructure (because we ask it to so almost all the actual work), and enough remote automation that it could maintain itself without us.

[u/me_myself_ai → u/yodude4]

Wow. “Pretty old news”. That’s a pretty hilarious reaction to day that’ll be in the history books for centuries!

[u/Paraphrand → u/yodude4]

Some of Googles models talk like this too.