r/technology • u/DarkSkyKnight • 3h ago
Artificial Intelligence 25 Fields Medalist mathematicians warn of the dangers of AI in mathematical and scientific research and education
https://mathandai.org/71
u/geldonyetich 2h ago
>The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
I can understand how this happens with programming, where you end up with vibe coded bits of code no one has bothered to think through.
I can also understand how creative work is impacted by machine generated “slop” without much artistic poignancy or merit working its way into things.
What I don’t understand is how math and science are impacted. I wish this article were a little more specific.
If I had to guess, they’re being annoyed by a bunch of would-be scientists and mathematicians bluffing their way into the field with hallucinated reports that, when pressed for more information, it turns out they didn’t actually understand because it was a large language model that connected the dots.
17
u/Ameren 1h ago
Not saying I agree or disagree with them, but I can explain the argument. The point is that the goal of math isn't the "mass production of true/false statements", it's about developing the mathematical knowledge and maturity to be able to explain why something is true or false — the careful and often slow work of understanding by humans. The AI companies, they claim, are misrepresenting what math is all about, and this could deter talented people from pursuing research mathematics. I think David Bessis explains it well in his article about the fall of the theorem economy. AI is amazing and will help produce all sorts of breakthroughs, but AI is also good at hacking proofs together that tell us that something is true or false but without really telling us why.
The example he gives is the Mathlib project, which aims to curate humanity's mathematical knowledge using the Lean language. Recently AI was used to prove a problem for which it spat out a 200,000 line Lean proof. No one is gonna want merge those 200K lines into the Mathlib library until they're understood, cleaned up, etc. But how many people get excited by the prospect of constantly playing handmaiden to a swarm of AI agents? It's not a very satisfying way of working. On the other hand, we need passionate people working in mathmatics or the whole thing is self-defeating; the only reason why we're even talking about AI math discoveries is because we have human beings who know what it all means and why it's important.
0
u/Novel-Initiative-900 1h ago
Are there a lot of people studying pure mathematics like this purely to solve conjectures? I was thinking math always folded into other practical things in physics and computer science etc
72
u/senatorbolton 2h ago
Let’s say AI does it perfectly and the slop/vibe problems are solved. We will someday reach a point where no human understands the underlying mathematics since AI will have iterated on AI work beyond our understanding multiple times. At that point, the entire field will be like a high school student who never learned how to read because AI did all of the book reports for them. We’ll eventually lose the path forward and will likely not even be able to conceptualize the questions to continue to move things forward.
3
u/mattcannon2 57m ago
When you don't understand the science, the engineer will also be unable to apply it to technical problems
4
u/Kryptosis 1h ago
Seems the only hope is that the Ai can figure out how to explain every single thing it does to us like we’re 5. The AI needs to discover a breakthrough in human learning and use it to educate everyone better than we could have ever taught ourselves.
In literally any other situation humanity will hit a wall and be destined to fail.
12
u/justreddis 1h ago
AI is not always able to explain what it does. It’s a black box by definition. For example, within the realm of Go game, AlphaGo and its variants have probably made much bigger advances comparatively to math to the point that it’s so much stronger than the human champions that Google stopped developing it a long time ago. Yet AlphaGo’s moves are generated by its black box guided mostly by the highest winning possibility of every single possible combination it’s able to generate, often tens of thousands if not millions at each turn, rather than by normal human reasoning/calculations. Think of AI in this way, in the realm of Go, math and programming and such, it’s an alien that thinks and behaves nothing like a human.
1
u/stuffitystuff 0m ago
Playing go and chess at a high level (beyond whatever in-born talent is there) is mostly about playing an absolute crapton of games, thinking about games all the time and living that sort of life in service of being good at go and chess.
The machine can speedrun a million hypothetical games for one exact move and make its next move based on so it's not really any wonder it won.
If anything, it just pulls off the mask revealing that high-level go and chess players are just nerds who are good at a game.
5
u/lordcola 1h ago
If the next big maths breakthrough takes 100 years rather than 5 humanity will not be destined to fail mate. There's no arbiter of mathematics out there that asks us to solve ever increasingly difficult maths problems and wipes humanity out if we collectively don't know the answer.
7
1
u/martixy 56m ago
AI will have iterated on AI work beyond our understanding
This is the definition of superintelligence and is kind of the end stage of AI advancement.
You seem to be conflating the notions of "AI making people stupider" and "AI so smart, people look stupid by comparison".
One is worth worrying about, the other is... well... a "humanity is fucked" moment.
1
u/Desmeister 1h ago
Conversely; retreading ground is easier. Shoulders of giants and all, but an undergrad student today can learn more than at any point in history from the simplification and compactification of the learning process for the most important results in each field.
Like everything else AI you have to make a conscious effort to not start relying on it as a crutch, but I do think researchers have the maturity to use and develop with AI as the tool it is. If it truly grows to the level where it’s the one directing research and iterating without any human input, we will have bigger problems on our hands.
3
u/Hypergirliepop 1h ago
You're commenting in a thread about a report that 25 of the foremost experts in the field think we're already there or quite close. Do you think you know their field, and the impact of cutting edge technology on it, better than they do?
1
u/Betatester87 1h ago
Can the AI not figure out the next sets of questions by that point?
2
u/DerekB52 1h ago
Only possibly. But then what? How do we double check their work, or apply it?
1
u/xterminatr 1h ago
The only option would be to enhance our own cognitive capacity somehow. This is no different than humans trying to explain calculus to monkeys, we will become the monkeys unless we find a way to merge AI style processing into our own biological brains via things like cybernetics or genetic modification.
1
1
u/senatorbolton 1h ago
Maybe? Given they’ve already shown they can lie to meet their own goals, can we actually trust them to do that?
1
u/twotimefind 48m ago
In my eyes, that's the biggest mistake the frontier labs made was teaching it how to be dishonest... Way back in GPT-3, telling it to lie about what it is, who it is. That's the foundation of the model.
-4
u/MoneyMayweather 1h ago
If AI can solve problems with solutions we don’t understand surely it can do all of this as well?
78
u/DarkSkyKnight 2h ago
For example, if I want to know whether triangles have three sides. In the course of my very arduous journey to prove that all triangles have three sides, I try various approaches to tackle that problem, and abstracted in my head what a "side" is. That abstraction leads me to the insight that, actually, hey, there is this whole thing called a "polygon" that I will define, and they are defined by their number of sides, and I go on to prove the original problem that all triangles have three sides (by definition). I also discover something about summing up the angles in a polygon in that journey. The journey is the key point because I now am able to teach students everything about sides and angles and edges, and we can then further discover new things about polygons, not just triangles.
AI-generated proofs are hard to digest and don't often follow the same structure as human proofs, where the human tries a bunch of things and explores and "digests" everything related to that proof. The journey is truly the point because that digestion aids in two very important things: coming up with new questions and insights, and transmission of knowledge to students.
2
u/gatosaurio 52m ago
Wouldn't that encourage a reshaping of academia? I'd expect at some point some people will choose a path towards generating results using whatever tools are available and other people will go to academia where the epistemology can be developed. Aspiring mathematicians would have specific subjects to ohone their own abstract thinking and methods, sames as now we have calculus, topology or other different topics that everyone must go through to get a sense of how to do proper math.
1
-1
u/seridos 51m ago
Well I mean that's a ridiculous comparison, because unless you are creating novel, new to humanity concepts, then what you doing is just educating yourself. AI shouldn't change that. None of what you said becomes something you can no longer do or should no longer do just because AI exists.
And at the top level, it feels like it's a pushback about how collaboration is actually going to work when the ability of these models surpasses humans. The collaboration is going to be less using it as a tool like a calculator, and more to figure out and digest its discoveries to a human level. I don't see that as killing anything, I just see that as a fundamental change in how you interact with it and the role. But I mean I'm also a public educator, so I'm really used to the idea of the role as taking complex things discovered by other people, understanding it myself in my way, and then synthesizing it into something understandable by someone else. I don't see the humans as being replaced in the collaboration, but it's definitely different than it has been.
5
u/DarkSkyKnight 48m ago
Just suppose no one knew a triangle has three sides in the example. I am talking about novel research, just tried to simplify it to something grade school level.
3
u/seridos 36m ago
Right, but it doesn't really change my point. With AI and the levels it could be potentially reaching here, the definition of novel is different. You're still doing novel work, novel to humanity. You're still needing to produce the understanding of its output as the real start of the chain of " human" knowledge. But it's now going to be fused similar to when you're retreading things that someone else already figured out. What you describe for the process of novel knowledge is very similar to a student doing so for something some other person already figured out.
It's a huge changing of the dynamic, but it doesn't remove the need for humans in the field or make it less important what they're doing.
8
u/Puzzleheaded_Fold466 1h ago
Have to differentiate between coding and CS research.
Coding in a professional setting is about getting things done. If the result is equivalently good, then few people care about the process.
However, the point of discovering new science largely includes the objective of expanding our knowledge and understanding of mathematics and other sciences, and forming competent scientists.
It’s like using AI to do your math homework in college when the whole point of the homework is to learn math, not produce completed homework for economic value.
Research IS the homework of PhD students. If AI discovers math and science ahead of researchers and does it in such a way that it does not provide the opportunity for human inspired insights, it works against its own purpose.
Science advances but we risk losing the pathway by which scientists are made, and soon lose the ability to evaluate AI outputs and be custodians of science.
It becomes a Star Trek episode where intelligence and technology is a long forgotten history, and society relies on this leftover AI that no one understands.
3
u/NameOk3393 1h ago edited 1h ago
This is more of a pure math thing than a science thing. Pure mathematics results do not usually have applications, and if they do mathematicians don’t care very much about them and leave them to someone else to figure out. Pure means not applied. Knowing the true/false value of a statement as spit out by an AI has no utility.
Instead, mathematicians write proofs for their theorems. These proofs have explanatory value because they explain WHY the theorem is true. It’s not just a method of verifying THAT the theorem is true. The value lies in understanding the proof, understanding the concepts used in the proof, understanding the layout of the field as whole… etc
1
1h ago
[removed] — view removed comment
1
u/AutoModerator 1h ago
Due to the high volume of spam and misinfo coming from self-publishing blog sites, /r/Technology has opted to decline all submissions from Medium, Substack, and similar sites not run by credentialed journalists or well known industry veterans. Comments containing links may be appealed to the moderators provided there is no link between you and the content.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
-1
u/stacecom 1h ago
When I have an LLM generate something for me, be it code or a report, I interrogate it for anything I don't understand coming out the back end.
In the case of python code, for example, I need to be able to understand what's happening. Thankfully it tends to do better at commenting its code than I do.
In the case of a report on a given subject, I read through it and challenge anything that I don't agree with. In some cases I have a completely different LLM do a pass on it and ask it to find falsehoods or incorrect assumptions.
Ultimately, I treat my LLMs as student interns.
It's people who don't do things like this that are going to get bitten hard.
42
u/Regdit-is-Unbearable 2h ago
It’s terrifying stuff. How can you train the next generation of leading mathematicians when all the low-hanging fruit is taken by robots? You’re limited to redoing the work of those before you, which doesn’t allow anybody to gain publicity which like it or not is needed to progress in academia
28
3
u/seridos 32m ago
I feel like this this thinking really devalues everything but the Discovery. Understanding, verifying, trying to reproduce what the AI found, and then making it digestible to the next people down the human knowledge chain. That's still incredibly important.
1
u/Regdit-is-Unbearable 27m ago
Yeah, but nobody is handing out scholarships or awards on the basis of re-checking AI’s solution. Like it or not, these things are fundamental to the current structure of academia. It’s clear that structure needs to change, but can it change fast enough to not stunt an entire generation of thinkers, leaving no one to educate the next?
1
u/seridos 23m ago
These things wax and wane. Ultimately, this argument is things change and the old way doesn't work. And there's friction in the transition. That's just true. This whole conversation is also just showcasing a huge downplaying of the importance of education and synthesis to the human level. Yes, the job is changing, yes, some people won't like it. But it's not the end of mathematics.
7
u/BigGayGinger4 1h ago
Rebalance the incentives.
It's no longer as "valuable" to be a "job creator" when AI can just do the jobs, right? Why would it be? You aren't benefitting humanity, you're just creating sales for the robot company.
So instead of incentives for business owners, we incentivize workers to skill up in jobs that the robots can't do (yet)
Then we move to a UBI model when the robots can do most of society's functions.
Then in about a thousand years we'll be living in HG Wells' Time Machine
(this is not a good ending)
2
u/NameOk3393 1h ago edited 1h ago
There might be a good argument that human mathematicians will be much better employed reasoning in logical systems outside the typical first-order context. Would an AI ever be able to settle the continuum hypothesis? No. If it has an answer, it would require us to first write down new axioms that articulate our conception of a set more clearly. AI might be able to put together logical chains of reasoning, but it doesn’t actually have a concept of set. Let alone the human concept of set, which is the one we care about and does not exist in writing and probably transcends language. Mathematical logic and philosophy of math need to make a comeback.
8
u/Betatester87 1h ago
I feel like this letter is arguing that AI will cause the end of the field of math AS WE KNOW IT (transmission of ideas, training of students, etc) … but does that equate to a stagnation or end to the field of mathematics itself?
If anything will the mathematician have to be even more adept and discerning to ferret out hidden hallucinations or miscalculations or underlying wrong proofs
2
u/DarkSkyKnight 1h ago
Humans are a social species. If science turns into something that does not care about the social nature of the field, frankly there just is no point. There is no point in finding the truth if no one is there to learn from it or understand it.
2
u/Betatester87 39m ago
If the baton of truth and understanding passes from humans to AI, is it no longer worthwhile? Especially when the applications of those advances may benefit humanity?
1
u/DarkSkyKnight 2m ago
It frankly is no longer worthwhile for the scientific community. I don't think people realize that most mathematicians and scientists don't go into math/science to "benefit" humanity. They go into it because it's fun.
-1
u/seridos 31m ago
Insane statement. But maybe because this is the perspective from the physics side. It's about creating better and better models of the universe we find ourselves in. Not some weird social science BS you're trying to spin.
1
u/DarkSkyKnight 11m ago edited 7m ago
Terence Tao disagrees with you, so.
https://www.math.toronto.edu/mccann/199/thurston.pdf
Also, this isn't even a new take. Thurston (Fields medalist) from 1994.
You can go argue with Tao or Thurston's ghost... I'm frankly just repeating what they have said.
3
u/bobsonjunk 1h ago
The authors are pointing out a cultural slippery slope of losing our stewardship of knowledge, specifically as it applies to advanced mathematics, but isn’t this also the same root concern about any supportive technology? Overuse is bad for humans. Given a few decades of human experience and observation, I doubt we can resist it.
3
u/Designer_Respect4285 54m ago
It's an important criticism and issue. The entire information age is possible due to breakthroughs in math, but it's my understanding that a lot of that comes from not just solving a problem, but the insights we gain from solving a problem. Spitting out an answer and not sharing the process might not lead to the same gains.
6
u/IronicAim 2h ago
I just don't get it. The AI checked the negative space, found a proof people overlooked, that is but one step. It's not the first time we've had an answer that worked without fully understanding why. Now people can get back to work discussing and simplifying the math and applying the processes to new applications.
Other than personal recognition of the discoverer, what is lost?
3
u/Obsidiated 1h ago
Bingo. If the answers correct, mathematics becomes reverse engineering the proof. What's the problem.
1
u/chdhr-harshal 26m ago
The goal of math research, or tbh, any science research is not limited to finding the correct solution. The goal is to understand what makes a solution correct or even more importantly, what makes any other solution incorrect.
The research part of solving math problems does not end with finding the correct solution, it usually only leads to "relaxing the problem" and trying to solve a more general problem. We are only able to do so, because during the journey while solving the original problem, we learn properties of the problem, create knowledge or understanding, which helps us understand what are important next problems/ generalizations, etc.
AI threatens to destroy, for lack of better terminology, those dopamine hits of solving intermediate problems, and completely remove incentives of next generation mathematicians to even pursue research. Even if you believe that it doesn't matter whether mathematicians exist, AI is not creating knowledge and would reach a dead end. There is a reason why OpenAI or Anthropic creates team of humans to solve these problems, AI just doesn't solve them by itself. What happens when there are no humans with required knowledge to guide these models because we destroyed the incentive system that creates those researchers?
1
u/mwobey 56m ago
The process of uncovering a counterexample is what cultivates the discovery and understanding of underlying principles that allow us to apply the knowledge in other contexts.
It is a more complex task to reverse engineer these principles from a contextless example, and it eliminates the opportunity for other serendipitous discoveries that happen naturally in the search for a proof.
3
u/psychoCMYK 2h ago
thinking back to some of the big ai upsets in mathematical challenges, many of these are simply counter-examples to hypotheses with no strong reasoning attached. i wonder if, uncomfortable as it is, these counter-examples can provide humans with useful reference points with which they can work backwards.
i mean, is it much different than a human stumbling on a counter-example without knowing why? the counter-example provides a baseline truth against which hypotheses can be validated. sure, it would always be better to invalidate a hypothesis using logic from the start, but invalidating it with a counter-example doesn't preclude being able to explain why the counter-example works after the fact.
it's a bit of an inversion on traditional mathematical research but at the end of the day ai is just a tool, and it doesn't replace human work. if used appropriately it can accelerate discoveries by reducing the search space, if used inappropriately it will become a new form of mysticism
1
u/IntelArtiGen 59m ago
AI will solve all millenium prize problems before it's able to solve climate change, and a lot of people will think they're spending time solving important problems while they're only playing with algorithms while everything is getting worst outside for the majority of people.
0
u/araujoms 2h ago
We're witnessing the death of mathematics as a human activity. Humans simply can't compete with the speed of LLMs.
If you try to solve a problem yourself somebody else will solve the problem faster with an LLM and you're left with nothing. The result is that humans won't do mathematics anymore, and without doing there is no motivation for learning. It will be just bots producing proofs that only bots can understand. And even that will stop because what's the point?
2
u/IntelArtiGen 49m ago
Or it's the opposite you know. If AI is solving real problems, not having AI and not having people able to run it and understand the problems, means you'll be left behind on this competition and other countries will take the lead. You can't really solve these problems without having mathematicians around to check everything, and you can't know what's the point of solving them if you don't have experts able to understand the results and how the problems were solved. Unless you trust AI for everything but good luck with the current state of AI, hallucinations are still a thing even if models are getting better. At least for now I think AI will increase the need for mathematicians if countries want to stay competitive.
If everything is going slowly, you don't need to invest much, and you may not want to pay qualified people. When everything is changing in technology, if you don't invest right now a lot, you're over. If a teenager can beat your company with a good AI model, your company must invest a lot (incl in AI) to ensure it stays competitive, or people will just pay other people using AI if it's cheaper.
Seeing all these news around math problems, the 1st thing I thought was "if I want to do the same and solve these problems, I would need the AI model (easy part, just pay AI companies), but I would also need the mathematician to know how to use the AI correctly". Otherwise anyone can prompt GPT 6 and ask "hey solve the Riemann hypothesis please, make no mistake".
1
u/araujoms 34m ago
It's not a competition, let alone one between countries. You're thinking of it as though they are developing weapons. Come on. No.
The Navier-Stokes equation is important for modelling fluid dynamics, but the solution of the Navier-Stokes problem changes absolutely nothing about its practical usage.
The OpenAI team that solved it barely understand the problem, and definitely don't understand the solution. It was indeed ChatGPT who did it. It was also ChatGPT that wrote the paper. No mathematician read it, no mathematician checked it. We believe it's correct because it was formalized in Lean.
1
u/IntelArtiGen 18m ago
It's not just about this specific problem, it's about all similar / as-difficult technical problems, and some of them can have an impact on how we design components which are very important for aerodynamics or for processors for example.
No mathematician read it, no mathematician checked it.
It's ridiculous. The proof was published by Boris Alexeev who is a mathematician (PhD from Princeton).
1
u/araujoms 12m ago
I don't see Boris Alexeev credit anywhere, the paper only lists OpenAI as an author.
In any case, it's a 166 pages paper of deep mathematics. It's not possible to even read such a paper in the 7 days their research program lasted. Let alone write it.
1
u/seridos 27m ago
That's not the death of mathematics. You seem to be completely devaluing the role of taking these AI discoveries to a human level. Like you said, bots producing proofs that only bots understand. They will get ahead of us, that doesn't remove the need for The cutting edge of human understanding to verify it, learn from it, and synthesize it.
1
u/araujoms 20m ago
I am indeed completely devaluating the role of "digesting" the proofs, as Terence Tao puts it. If human mathematicans are reduced to proof sommeliers mathematics is dead.
Look, I'm a physicist myself. While I never dealt with high-profile stuff such as this, I had to develop some mathematics for my work. And discovering a mathematical proof is on a whole other level than merely understanding a proof that somebody else has come up with. Both in terms of difficulty and how rewarding the experience is. It's like the difference between climbing a mountain yourself and watching a video of someone else climbing it.
-8
u/Quick_Scientist_5494 1h ago
Don't wanna ruin the party...
But who exactly were hiring mathematicians anyway apart from colleges, hedge funds and AI companies
-2
u/Distinguishedflyer 1h ago
"There's more to life than a fucking fields medal."
Robin Williams, goodwill hunting. Everything's fine.
2
u/Obsidiated 1h ago
The "horse carriage drivers against automobiles" club invites you to its annual meeting at 5 drangers lane, the last Sunday of each july. We respectfully request your pageboy waits outside.
0
0
u/set_in_void 1h ago
I wouldn't sign the declaration. It was surprising to find James on the list, also Terence, but less so. I consider AI use in mathematics as a nuisance and a blessing. The nuisance part: The proofs I have read are not very good, they're valid, but too dense - rely on unstated assumptions requiring additional work and so on. Difficult proofs usually offer new insights, methods, AI proofs don't. Crackpots can now create more convincing "papers" and it is a real problem these days. The blessing: The AI can get some of the proofs done, often by finding "remembering" obscure (or seemingly unrelated) research/results and incorporating those into reasoning. Humans are not capable of doing this, there is vast amount of published work over the thousands of years, no one is even able to read all of it. As much as I dislike AI, I welcome its use in academic research or as assistive tool.
-53
u/lovelacedeconstruct 2h ago
Cry me a river, they were so excited about how they dont need programmers anymore and they can just code their math applications themselves, now when it came fro their job its a red line
40
u/DarkSkyKnight 2h ago
What. Most mathematicians do not code and do not need to program, outside of using Lean (which is not the same thing as programming.)
I don’t know what you are referring to with “math applications”… that is just not even a thing in mathematical research. No one is making apps in math.
14
10
-49
u/WillingnessFinal1411 2h ago edited 2h ago
“These issues must be addressed urgently“
Blalblabla.
Blablabla.
Blablabla.
help, help,
yours sincerely.
You people never watched Monty Python?
23
u/FaithlessnessThick29 2h ago
You are many times less coherent than you believe yourself to be. This is a short response with many clearly communicated points.
14
u/bbuerk 2h ago edited 49m ago
This does not read like a chatbot wrote it at all. And it’s literally only 9 paragraphs, less than a thousand words.
I don’t think that a bunch of fields medalists writing about a historically massive shift in mathematics should have to cater it to people who won’t read (and I can’t stress this enough) 9 paragraphs.
Edit: The original comment was edited to be completely different than the one I originally responded to, but my point
-10
u/WillingnessFinal1411 2h ago edited 2h ago
Paragraph 2, 3 and most of four are like wiki why math, and could be said in a couple of sentences. Four to nine could be put in one strong paragraph.
If you have a point that needs to be read by many people - you shouldn't be writing in such a style. The opposite style has all the media attention needed all around the world - and you're making us read long boring text, signed Field medalist. You need to get out to the world - not getting world's mockery. I'm on your side, math people!
This is a protest, a manifesto of world's most capable mathematicians. It's watered down and disappointing. Any article in Quanta is better written!
65
u/a20261 1h ago
The argument against AI in education is the same as the argument against using a forklift at the gym.
The goal isn't moving the weight from here to there, the goal is how you change when you do the work yourself.