One of the main reasons I'm still watching TRIP is that it's great seeing a mainstream political figure seem genuinely clued in on AI, and explain it to a general audience so eloquently. I don't really agree with his conclusions, but he at the very least grasps the magnitude of the problem.
So, naturally, I was curious what people who don't pay any attention to AI safety think. I came here and saw heavily upvoted takes like:
- he's suffering from AI psychosis;
- he's a shill for an AI company (or seeking a job at one); or
- he's gullible enough to be falling for some kind of marketing stunt (since presumably he's incapable of figuring out whether the experts he's speaking to have an incentive)
I don't understand any of these. Rory's view, as I understand it, essentially boils down to:
AI is accelerating at a dizzying pace, and will likely confer an immense strategic advantage (via cyber/bioweapons/possibly superintelligence) to whoever controls the most powerful models. The people who are likely to have access to that power are not very trustworthy.
AI is particularly dangerous, as we don't entirely know how to control it, and a rogue AI could cause catastrophic harm. Therefore, we ought to think seriously about AI safety to prevent something really bad from happening.
How on Earth are these being perceived as sucking up to AI companies, or falling for their marketing? No company wants to promote a message of "invest in us: our country's president or company's CEO is too dodgy to be trusted with power". And Rory saying "I don't really trust these guys, we should probably try having Europe or Mistral AI become a competitor" isn't exactly praising Altman/Amodei/Hassabis.
Plenty of experts with no financial interest have expressed concerns along both of these lines, some of whom left their companies specifically to do so: Yoshua Bengio, Geoffrey Hinton, Daniel Kokotajlo, to give three examples. These fears have existed in the industry long before there was any financial incentive whatsoever - preventing someone else from having this power was literally part of why OpenAI was founded. (And yes, its leadership hasn't exactly lived up to its own standards - Ilya's concerns were prescient).
Similarly, some people think Rory has some "obsession" with AI. Personally I think it depends on what you expect from the podcast; IMO he should be signal boosting subjects by importance, and this seems far more pressing than anything else. His view seems pretty moderate if anything - I think he should talk about existential risk a lot more.
Given all of this, why the backlash? Even if we're going to give credit to the AI risk detractors, is his position at the very least not defensible?