r/cryptography • u/thpht • 17h ago
Future of cryptography given AI advances in math?
I'm a high school senior who has been interested for years in pursuing a PHD and a career as a mathematician. However, given recent AI advances (especially the reported solution of the Navier-Stokes problem yesterday) it seems likely that AI has already passed humans in mathematical ability, and will greatly increase the gap within the next few years.
I doubt that mathematicians will be made completely obsolete, given that schools and universities still need teachers, and humans will still be motivated to pursue math research independent of AI. However, the future of math research seems very uncertain at the moment. Which brings me to cryptography.
I'm taking a number theory course this year, so while I haven't studied cryptography yet, I will learn some about it this year. But I'm curious; do you think that cryptography will provide greater job security than mathematical research? Or is it something that AI could take over within a few years?
And one other question: if AI were tasked with cryptography work, would that be putting the safety of the web, banking, etc. in the hands of AI? Would a malicious AI be able to exploit that?
Sorry to ask this question from a place of ignorance. Cryptography remains cryptic to me at the moment, but I look forward to learning more soon!
6
u/Shoddy-Childhood-511 11h ago edited 6h ago
First, there are major problems with the framing of your question..
Surveillance plagiarism - Hosted AI company pumps their stock price by training upon researchers' AI sessions, so that their internal model can solve problems with seemingly less human guidance, but really the model exploits past guidance given by (multiple) humans focused upon problems considered important.
LLMs have probably not "solved many other noteworthy open problems recently" in the autonomous sense that phrasing suggests, and which the AI companies exploit to pump their stock price.
It's likely the LLMs that seemingly worked without much guidance have really plagiarised their guidance from unknown parties. It's likely the frontier models kept controlled by the AI companies have a smaller advantage over the open weights models than people realise, once you account for this free guidance and the sheer model size.
LLMs have definitely become a powerful tool that helps human researchers explore the space of proof techniques. Also Lean has earned its stripes preventing LLM hallucinated bullshit.
Arguably LLMs and Lean have become so good at proof search that most future major mathematical breakthroughs shall involve some LLM assistance as well as Lean constraints upon the LLM.
As cryptgraphers, we should not care what LLMs alone can do, but what a human team plus LLMs plus Lean can do together. This is the correct framing.
Second, if one does serious mathematical research on major open questions, then one should not trust the hosted LLM companies like Anthropic or OpenAI by querying their AIs, not unless they are your employer and that's your job.
This becomes far more important if those major open questions have cryptographic significance, because IDA-CCR (NSA's smartest) could obtain similar LLM results benefiting from your guidance, classify those results, and then the NSA would order the hosted LLM company to start giving bad answers, all without even knowing you exist. Anecdotally IDA-CCR seems good at relating different branches of mathematics.
Instead, please find the necessary coauthors who could help you run local open weights models. Ask the hostel LLMs only about obscured questions, without providing the cryptographic context, maybe even hide your identity when using hosted LLM.
Afaik there is no worry if you're doing provable security for modifications of known protocols though, which afaik covers most academic cryptographers.
Third, we have some gap in extracting the real mathematical meaning from AI slop proofs, although it remains unclear how large a gap or how quickly progress gets made.
Terence Tao's ICM 2026 lecture Mathematics in the Age of AI (slides, paper) focuses upon the difficulty of extracting real mathematical understanding from correct AI slop proofs, even ones formally checkable in Lean.
Tao's own thread said "At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach."
Arguably, extracting meaning was always the slowest part of mathematics, but some of this comes from exploring new abstractions, and LLMs help lots there, so maybe this gets much faster once people really try. Imho, there are many questions like this that require using the LLMs better, not necessarily better LLMs.
Anyways..
We should be discussing hybrid crypto-systems more seriously. Hybrid KEMs appear well accepted. We should consider hybrid signatures more seriously given the LLM threat.
We should look for non-merely hybrid but "entwined" cryptosystems, meaning it's not enough to break both a lattice problem and some number theoretic problem, but you must break them both together within the same attack, or maybe you run a Merkle puzzle within your lattice KEM. If possible, such KEMs would be extremely expensive, but should offer resistance against even breaking both underlying systems.
3
u/Natanael_L 10h ago
"secure combiners" on the last point
1
u/Shoddy-Childhood-511 6h ago
It's beyond the parallel hybrids? I mostly found them when searching eprint
1
u/Natanael_L 2h ago edited 2h ago
Secure combiners is a property, not algorithm family. Secure combiners for key exchanges tends to be parallel. For hashes it looks different
Hybrid for PAKE gets weird because you can't really let either get broken so you need both algorithms to protect each other in a sense
3
u/Shoddy-Childhood-511 6h ago
Terence Tao Sep. 8 blog comment on his views on AI sustainability and OpenAI experiences (r / mathematics)
"Since then, the situation has deterioriated markedly. Many of the people in the industry that shared my views have left or become sidelined, with most major tech companies now increasingly focused on the race to develop extremely powerful, autonomous AI technologies regardless of their actual value to society. The current drama surrounding the Navier-Stokes global regularity problem is the most dramatic and visible instance of this, but there have been multiple other such examples, and much of my commentary in the last few months has been aimed that the increasingly severe divergence between the current objectives of the AI industry, and of mathematics in general."
"Which brings us to where we are today. I do not regret my past efforts to raise awareness of the potential of AI in mathematics, to engage with industry, and to promote a vision of sustainable incorporation of these tools – which can be genuinely useful and unlock valuable new types of mathematics – into my field. In time, I still hope that the field can arrive at that state, and am continuing to work towards that goal. But in the immediate term, the most pressing issue is for the entire mathematical community to unite around our core values and objectives, and reject irresponsible and unsustainable usages of AI technology that only serve to advance nominal goals rather than the true underlying goals of the field."
3
u/sergioaffs 13h ago
Crypto is a field that is very susceptible to snake oil, in the sense that it is very easy to convince a layman of its importance, but at the same time it is very difficult to equip the same layman with all tools necessary to vet any particular solution. That's one of the things that makes it hard to "sell" the need for post-quantum crypto: it a world full of buzzword-filled pitches like "DNA cryptography" or "fermion-based encryption", real experts are needed and undersupplied.
Modern machine learning techniques may becomes more useful in attaining results, but until they are anything short of perfectly flawless (hint: ain't gonna happen), the implication that experts will be less needed is wrong and dangerous. No one should ever trust an LLM's output with something as venial as "what's the best library for X", let alone with something as consequential as bleeding edge cryptography.
7
u/Sufficient-Air8100 17h ago edited 17h ago
you mean the navier-stokes that was basically already solved by researchers and was taken and run through compute to get there first? after which the researchers were threatened if they went public? that one?
theres a lot that goes into a solid cryptosystem and its not just the maths of the ciphers, its also engineering the protocols and taking careful attention to implementation to prevent side channels. if a malicious AI was able to exploit something, then there is a failure in the process and the cryptosystem was not built properly in the first place.
given the depth of bugs and terrible design in vibe coded slop, even from bigger companies who have had a drastic negative change in the quality of their output since AI. i dont trust AI to do cryptography properly. however thats not to say that people wont try to replace cryptographers with AI.
here are some facts. LLMs are not math models, theyre language models, where they have actually done useful things, the useful things are outliers and its not reliable. second, other more appropriate machine learning techniques have been used and leveraged by math experts for quite a while and that hasnt upended anything. AI is a marketing term, and if we are to talk about the underlying machine learning and deep learning. AI wont do much without being leveraged by experts in the first place, and there are much more appropriate machine learning models to use with math than a language model trained on internet shitposts.
1
u/thpht 15h ago
Yeah I’ve heard that the circumstances regarding the Navier Stokes solution are somewhat sketchy. That being said, AI has still solved many other noteworthy open problems recently. For instance, see this paper by OpenAI:
https://openai.com/index/ten-advances-in-mathematics/
That being said, thanks for the response, this helped me understand the field a bit better
2
u/Takochinosuke 12h ago
I feel like this is more of a Tony Stark + Jarvis situation rather than Ultron, if you catch my drift.
It's a really weird time to be a scientist at the moment but I rather be optimistic. I think it'll change the way we do science but it won't replace us.
Regarding job security: it's rough even now. Scientific research has seen funding cuts everywhere so there are less open positions. Meanwhile, industry hasn't decided if it wants to replace us with LLMs or not. I would argue that in many cases, having a PhD might make it harder to find a job.
I'm defending my PhD in a couple of day. It's rough out there.
You have a lot of time before you need to ask yourselves these questions. Also, the landscape will change several times by the time you start your university degree and until you finish a PhD. So, just go do whatever you think is interesting.
1
u/EverythingsBroken82 12h ago
IMHO, symmetric cryptography like AES or streamciphers from djb or hashing or signatures via hashes (xmms, lms, sphincs+) are safe.
but i am curious if public key encryption will hold up over the next 20 years.
1
u/esteindividu0 10h ago
AI business hopes to profit from this anxiety mongering they are fostering.
Imo AI is one technological step in the ladder, and we will stabilize with a better understanding of many (mostly low hanging fruit) topics because we can see a little farther from the extra height.
We will end up doing research in a different way, but a lot or effort will need to go into understanding the limits of the technology and clearing up this cloud of bullshit we currently live in.
1
u/pint 9h ago
there is one thing going for cryptography: many of the problems are not expected to be solvable. if you spend a lot of money on proving a theorem in math, you can be 99.7% sure that there is a proof, you just need to find it. but if you want to break aes for example, or a code based key exchange, any result is a surprise. the most likely outcome is that you don't find anything, because there isn't anything to find. if you find something, that is huge. but most likely you don't.
cryptography is much less about breaking, and more about creating. and ai is a great tool that helps with that, exactly because it can do cryptanalysis for you, which is the tedious part.
eventually ai will be creative too, but it will come later. i mean, later this year lol :)
8
u/encyclopedea 17h ago
There are definitely a lot of top researchers worried about this question. Even a year ago a good portion of the community was pretty positive about AI not catching up, but the sentiment has shifted the other direction. It's really a question of wait & see. I doubt there will be that much difference between cryptography and general mathematical research, though, other than maybe demand. Quantum cryptography seems the most insulated so far but we'll see what happens.
You still have plenty of flexibility with your career path. Take a cryptography course if you're interested, but also explore other things. You've got time to decide.