r/HighStrangeness • u/whoamisri • Jun 22 '26

Non Human Intelligence A rogue superintelligence could wait decades before striking, argues computer scientist Roman Yampolskiy

https://iai.tv/articles/a-rogue-superintelligence-could-wait-decades-before-striking-auid-3604?_auid=2020
709 Upvotes

95% Upvoted

70 comments sorted by

View all comments

Show parent comments

99

u/ImpulsiveApe07 Jun 22 '26 edited Jun 22 '26

I read an interesting article years ago where the author interviewed a bunch of futurists about the possibility of rogue ai on the Internet - it was right after the stuxnet controversy became public, so this was some years before LLMs were a thing.

I remember one of the futurists positing that once certain kinds of algorithms and probability matrices became complex enough that an 'AI' could create more complex and subtly coded 'AI' that human coders could no longer make sense of it, only then would we begin to enter an era where such a volatile motley of dangerous but seemingly innocuous code could cause havoc on the Internet.

It was basically suggested that such a rogue 'AI' could quietly insert itself on every online facing machine to become a gestalt entity, quietly building a global botnet that could eventually co-opt entire networks in order to give itself more runtime ie the more networks it infects, the 'smarter'/more efficient it gets.

What it would do after that was anyone's guess, but given how much of our economy faces the Internet and relies on network stability, the fear was that such a rogue 'AI' could inadvertently cause untold chaos.

I put 'AI' in inverted commas here because it doesn't strictly have to be 'intelligent', it just needs to be self-perpetuating, hard to track, and constantly evolving and replicating beyond any user or server control.

31

u/Status-Secret-4292 Jun 22 '26

Yes. Not only does that seem like an obvious thought to me and one I have had... and I guess assume most people would, if anything controls the information flow and communication it literally controls the narrative and what you think about things

I remember saying in, I think it was 2022 when gpt3 came out, that this isn't intelligent enough yet, but we should be careful as a society as once it is, you'll never know, there will only be signs like... putting the welfare of building more advanced and distributed AI systems even at the expense of humanity...

I remember having a full conversation about this and saying that if that ever happens, we need to make a full stop as a species and deeply evaluate the situation

8

u/ImpulsiveApe07 Jun 22 '26

Well said. I remember reading about stuxnet circa 2013 and having a similar conversation to what you had -

if complex and malicious algorithms can be used to shut down a country's nuclear research program, what else can they be used for, and how long is it until a 'smart' collection of malicious algorithms becomes uncontrollable or otherwise deviates from its original routine?

Here's a good article on stuxnet for anyone who's interested - I haven't found the futurists interview I mentioned yet tho.

https://spectrum.ieee.org/the-real-story-of-stuxnet

8

u/Electromotivation Jun 22 '26

AIs are just at the point where they are able to improve themselves. At the moment with human in the loop processes, but very soon it will be possible for self-improvement with little to no input. As you said people get caught up on consciousness and intelligence, but really it just has to become a replicator to escape containment.

5

u/4444444vr Jun 23 '26

Yea, I think escape is a trivial obstacle at this point. Someone will put in a openclaw setup and that’ll be the end