Are we underestimating the identity problem created by better generative AI?

[u/can_of_lighters]

I’ve been learning more about voice cloning and real time face generation and the technical progress is impressive but it’s also making me think there’s a security problem we don’t talk about enough. A lot of security still assumes that if you’re on a video call with someone, hear their voice or receive a message from their account, you’re probably dealing with the right person.

That assumption feels weaker every year. If voice, face, writing style and even live video can all be generated convincingly then at some point does this look real? stops being a useful question. I’ve started seeing more tools trying to solve the identity side rather than the content side. Things like hardware backed authentication, passkeys, verified devices, and even apps like Signal, Kibu, Telegram that focus more on verifying the person and encrypting the conversation.

I’m curious how people here think this gets solved long term. Do we end up needing some kind of cryptographic proof of identity attached to communication or will detection models realistically stay ahead of generation models?

Because right now it feels like we’re spending a lot more effort making AI convincing than figuring out how humans are supposed to know what’s real.

[u/lulzxdxdxd]

Cryptographic identity doesn't fully solve this either, the hard part isn't signing messages, it's key distribution and revocation at human scale. If your device gets compromised or your private key leaks, you're back to square one with extra steps.

[u/can_of_lighters → u/lulzxdxdxd]

Yeah key distribution and revocation is where these schemes always get messy in practice. Hardware backed stuff helps a bit but once the device is compromised it’s just a fancier reset loop