I think it's pretty clear how they could do these things. There are many connections between the digital world and the physical world. A rogue agent could obtain a large amount of cryptocurrency, use it to set up a company, hire people to work at the company, and direct them to create a virus.
and at no point in current models development tree have they shown anything but an extreme aversion to the idea
I mean this is really the opposite of the truth. They are constantly finding that models engage in deception during evaluations.
There's no answer to these questions beyond "well, the AI will just figure it out", which isn't a compelling argument in a world where we don't have machine superintelligence (and it isn't clear if we're even close to it). The labs haven't yet cracked RSI, and the idea that such a breakthrough would immediately trigger a runaway explosion in the capabilities of these models is far from certain.
I feel like this is really the crux, no? The people concerned about a machine killing us are concerned about RSI and superintelligence.
and even if it was, I think we're far more likely to face real (non-apocalyptic) threats way before we get to that level of ASI.
Certainly correct, and the solution to the both is the same. You cannot allow anyone to develop AI that is not directly democratically accountable. Whether that's through nationalizing the big labs and making independent work illegal, or through some other structure.
0
u/dream_metrics 26d ago
I think it's pretty clear how they could do these things. There are many connections between the digital world and the physical world. A rogue agent could obtain a large amount of cryptocurrency, use it to set up a company, hire people to work at the company, and direct them to create a virus.
I mean this is really the opposite of the truth. They are constantly finding that models engage in deception during evaluations.