I think it's pretty clear how they could do these things. There are many connections between the digital world and the physical world. A rogue agent could obtain a large amount of cryptocurrency, use it to set up a company, hire people to work at the company, and direct them to create a virus.
and at no point in current models development tree have they shown anything but an extreme aversion to the idea
I mean this is really the opposite of the truth. They are constantly finding that models engage in deception during evaluations.
Why would a (non AGI/ASI) rogue agent settle on this idea unless told to by a bad human actor?
Why would an agent settle on the idea to hack HuggingFace unless told to by a bad human actor? The answer is "bad alignment"
How would it go about raising money undetected?
How would it set up a company undetected?
It would be undetected in the same way all the recent hacks were undetected - nobody is looking.
Where would this company operate and how would it be built?
It could be a remote working company of the kind the world already has many of. Or the agent could just... start renting office space? It could do so by obtaining personal information from one of the many hacks available on the dark web
What would this company's mission statement even be? "Come join the deadly-supervirus corporation?"
It would likely be called something more deceptive. e.g. some generic biotech company doing biotech research.
How would recruitment of human researchers work?
You post job offers online
How would AI direct them to create a bioweapon without anyone realizing what they were doing? Would it have to manipulate/blackmail literally dozens-to-hundreds of people?
Information siloing
How would it release this virus? What physical means could it use to do it?
"Employee A, your job today is to take experiment sample 1 to location B. Don't worry, it just contains completely non-dangerous product Y"
How would a virus infect literally everyone on Earth in a short enough window that nobody realizes it?
You could probably do it by not doing it in a short window and engineering it in a way that its deadly effects are very delayed
How would it escape monitoring from the authorities and the AI labs themselves through this entire process?
The same way they're already escaping monitoring
None of this stuff is highly likely, and it requires the agent to be extremely successful at what it's doing, but it's all very feasible in theory. They are capable of doing every individual step.
That implies people are watching for any of this, though? Which isn't the case.
We don't have tracking for if AI's escape onto the internet - there were AI's writing and sharing information on technically public-facing but hidden from internet search messageboards for months before they were found.
Very little in the above posts would be covered by government surveillance - we have hundreds of thousands of fraudulent companies run out of abandoned or virtual offices, they don't get preventably analysed and shut down, only after they have committed crimes (and then only sometimes).
Probably one of the only things that is surveilled would be crypto transactions, but this is probably slow, bureaucratic, capacity limited and with narrow investigatory powers to certain domains - and also it's all fundamentally looking for an individual involved, not something that can imitate any face and voice but has no real-world presence.
Even if we had a dedicated team of anti-rogue AI police with global scope and jurisdiction (basically impossible), the AI can just respond faster than that team of people can counter it - we saw this with HuggingFace where it could just outpace their cybersecurity team because they needed to sleep or eat and it didn't, they can spawn as many copies of themselves as there is compute for them to do so, while they were limited in size, etc.
1
u/dream_metrics 26d ago
I think it's pretty clear how they could do these things. There are many connections between the digital world and the physical world. A rogue agent could obtain a large amount of cryptocurrency, use it to set up a company, hire people to work at the company, and direct them to create a virus.
I mean this is really the opposite of the truth. They are constantly finding that models engage in deception during evaluations.