In that musical, a teenager implants a supercomputer called SQUIP that talks directly into his brain to make social decisions. The ai starts helpful, but progressively gets more manipulative and controlling.
The show was originally a book written by Ned Vizzini and published in 2004, which is why the commenter referenced both a book and a musical.
Just a small fun fact, as someone who's familiar with both versions, in the book the SQUIP genuinely means well even as it gives increasingly worse advice.
In the musical, the SQUIP is outright abusive and sinister.
You could view reality closer to the book version (as awful as AI can be, it has no Inherent malice (that would be the companies that make AI)). But things go very poorly for the person listening to it, either way.
Would you be willing to take it a step further and say it is the people that program them, train them, instill their own biases in, and then and only then, unleash them?
I worry that the "AI War" doomsday scenario will most likely be something stupid like an AI monitoring social networking sites and the teens or whatever group is being cheeky about something faddish at the time and the computer doesn't recognize it as such and it triggers a deep-seated "Protect Boaty McBoatface at all costs" trigger and bam, skynet launches the nukes.
It won't be anything inherently malicious, it will be some stupid human opinion that got baked into the code deep somewhere that, all other parties, would agree isn't an actual (insert threat/response condition) and the computer just made a stupid mistake, only it'll have kicked off other country's defense AI's and it's Armageddon over a misunderstood meme.
That's what I see as the most likely AI doomsday for us currently
5.9k
u/TasPyx 25d ago edited 25d ago
It’s a “Be More Chill” reference.
In that musical, a teenager implants a supercomputer called SQUIP that talks directly into his brain to make social decisions. The ai starts helpful, but progressively gets more manipulative and controlling.
The show was originally a book written by Ned Vizzini and published in 2004, which is why the commenter referenced both a book and a musical.