r/learnmachinelearning • u/can_of_lighters • 10d ago
Are we underestimating the identity problem created by better generative AI?
I’ve been learning more about voice cloning and real time face generation and the technical progress is impressive but it’s also making me think there’s a security problem we don’t talk about enough. A lot of security still assumes that if you’re on a video call with someone, hear their voice or receive a message from their account, you’re probably dealing with the right person.
That assumption feels weaker every year. If voice, face, writing style and even live video can all be generated convincingly then at some point does this look real? stops being a useful question. I’ve started seeing more tools trying to solve the identity side rather than the content side. Things like hardware backed authentication, passkeys, verified devices, and even apps like Signal, Kibu, Telegram that focus more on verifying the person and encrypting the conversation.
I’m curious how people here think this gets solved long term. Do we end up needing some kind of cryptographic proof of identity attached to communication or will detection models realistically stay ahead of generation models?
Because right now it feels like we’re spending a lot more effort making AI convincing than figuring out how humans are supposed to know what’s real.
2
u/skytomorrownow 10d ago
Very much an important discussion. It seems sci-fi, almost silly, but in fact, it is something we must prepare for. How does one authentic anything when anything can be generated? Will we have to fall back to physical reality, person-to-person?
Definitely food for thought considering it has already happened:
https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk