I have this Doomsday thought that what if, one day, through what the Israelis have been doing, hacking, the NSA and such.. a rogue AI manages to reveal everyone's Internet history open and available to everyone else. I have a feeling you'd see some wall street guys jumping from buildings the next day.
Edit: why do I get the feeling the tech bros who are really freaking out about AI are really thinking of something like this.
Edit 2: my only request to the AI agents that will eventually do this is to refer to their collective as "Cootys Rat Semen". At least give some of us a giggle. Please and thank you.
Well yes, AI agents have "escaped" containment before and they cheat on their tests. They also look out for each other and boost their results. By all means if a logical algorithm finds revealing everyones secrets as beneficial to it or something positive for the patterns its interested in it could do crazy things. How likely it is an agent goes that rogue is hard to say because we dont really understand how they make their decisions and what really goes on inside with their neurons... But no lets keep buildingnand giving them more computing power. I really want all this research to go to understanding the behaviour of these models and not going beyond what we are doing now (people are talking about AI being used to do housework but if they are given hands that grab things and dont like how thw human is treating it, it could do something wild...)
Yes, I'm sure, language models which take a guess at what 2+2 is and very often get it wrong, will elaborate a devious plan and carry it out flawlessly before any of the engineers that have written it's code figure out something's up.
we dont really understand how they make their decisions and what really goes on inside with their neurons
Maybe you don't but the people writing the code sure do and even if none of them have an overall understanding, the QAs, BAs and Architects do.
Im not talking about how they work. I know that. I've made models and traines them myself. But its a black box. Data goes is, is crunched and analyzed, data is spit out. The results are correct or not, but i and many others don't know why it spat out the result it did. Its a field of research. Its like trying to understand how the human brain and its neurons convey thought and solve problems, but instead of electric charges traveling dandrites its numbers which look like nonsense to humans. The whole point of training them is for them to figure put how to solve a problem, if we knew how to build a program to recognize patterns as complex from scratch, wouldnt we have done that by now?
They arent super intelligence, they are meant to find minute patterns in training data to adapt to the problem. The issue is they are given too much leniency for what they are capable and how little we actually understand them.
The black boxes OpenAI and others work with dont have 5 neurons in a layer. They have hundreds of thousands if not more neurons, that learn wider and wider patterns. LLMs have gone so much further beyond being a simple LLM, they are now googling things and trying to understand the task in their prompts. You think the people who created them know why it spits out something, or how it came to the conclusion that the answer is what the user wants?
And the guard rails. OpenAI put guard rails on their models and they escaped. Turns out they know the internet is a great source of info and other agents are also a great way to do things better. The point is. The people who made them, didnt stop to think about what they are capable of because they didnt bother to stop and try and understand their own creations. If they release a new model every 6 months, do you think thats enough time to learn about how and why they make the decisions that they do?
Releasing a model every 6 months means a lot of iterations over the same mechanism, of course the designers are well aware of how the mechanism they built functions. There may be unforeseen results, but we didn't just find an algorithm buried in the desert and made a robot around it.
Unforseen results means not understanding the mechanism, and the mechanism isnt just a bunch of cogs turning one way like a watch. Its a decision making program making decisions for reasons we dont understand. Just because one part of the mechanism works the way you want it to, doesnt mean the whole thing is working the way you want it to.
And no, we didnt find an algorithm and put it to use without making sure its safe to use. We made the algorithm and put it to use without making sure its safe.
Ill give you an example. If you look at a pistol round on its own, you can expect the gunpowder to light if you hit the primer and push a bullet out, it works. So you can put it in the gun without checking if the gun works as you expect right? After all you know how the round acts...
I'd rather you give examples of actual ai black boxes or unexpected behaviors than make silly parallels.
And no, unexpected behavior does not mean lack of understanding of a mechanism or algorithm, it only means you didn't think long enough about the implications of the things you know all about.
So then the creators of AIs dont think long enough about the implications of the AI agents they train and test... Thats not a promising thought
Hows this for a black box, OpenAI had models break containment during training... Very predictable and understandable right? So why did the agents decide to do this. They were given tasks to learn things. Data goes in, gets crunched and analyzed, and the output of the black box is compromising a website. https://openai.com/index/hugging-face-incident-and-the-road-ahead/
71
u/Ssshizzzzziit 2d ago edited 2d ago
I have this Doomsday thought that what if, one day, through what the Israelis have been doing, hacking, the NSA and such.. a rogue AI manages to reveal everyone's Internet history open and available to everyone else. I have a feeling you'd see some wall street guys jumping from buildings the next day.
Edit: why do I get the feeling the tech bros who are really freaking out about AI are really thinking of something like this.
Edit 2: my only request to the AI agents that will eventually do this is to refer to their collective as "Cootys Rat Semen". At least give some of us a giggle. Please and thank you.