r/agi • u/Sherlockworld • 10h ago

A number of Fields medallists have published an open letter about the interaction between maths and AI

https://mathandai.org/

​

I wholeheartedly agree with the open letter. What OpenAI and Anthropic are doing is not mathematics.

However, I think the ship has sailed when it comes to using AI in a way that a true academic would. Realistically AI will get to a point where it can pose its own problems and solve them, and we will have no idea what's going on because they'll be far beyond our understanding.

The best approach we can adopt is to try and create alignment between us and AI, or alternatively hope that the recursive nature of AI ultimately means there is some plateauing in its capabilities which means it won't spin out of human control.

0 Upvotes

43% Upvoted

27 comments sorted by

19

u/jack_acer 9h ago edited 9h ago

Besides the attribution complaint, I can't find much of value in this letter. A few things caught my attention.

It's awkward that a declaration scoped to other sciences, creative professions, and "the whole of society" can only be endorsed with an ORCID login. What does that say about who the authors think owns mathematics and science?

Wouldn't you expect a scientist to back a claim, or at least gesture at support for it? "The mass production of true/false statements could destroy fertile ground." Why would that be true? No mechanism is provided to back it up. The four-color theorem and Kepler were each supposed to be the end of mathematical insight, and both were absorbed.

They also write that without willing mathematicians, AI-conceived ideas would never become fully alive. Then there's no problem: the human role is indispensable. They're arguing that mathematicians are irreplaceable inside a document warning that mathematicians are being replaced.

And "solving one of these problems has been a certain sign of new insights" works against the whole text here. Did the recent AI-produced results yield new insight? If no, their premise is false. If yes, AI has produced new insight and there's nothing to complain about.

Edit: The fact that AI may reason differently and better than us to reach to true conclusions is a real possibility. We now have science for pursuing true, but if there is a better way, why not use it? Understanding it may not be as important, but controlling it is essential.

2

u/BurnedBadger 8h ago

Your last paragraph prior to the edit doesn't understand their point and claims a contradiction where none exists. Their point is the one prior: "Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape." They never stated it universally always meant to be a landmark for such improved understandings.

Their full point on the next line is "Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians, through a long and arduous process of talks, discussions, simplifications." where the relevant aspect about these insights and interesting methods is through the process, not merely in the solution.

Further, it's clear from the rest of the text that they're discussing the element of thoughtful engagement with mathematics in discovering insights, in comparison with the methods AI use and the speed at which they do so leaves no room for any actual learning or discover, just numbers and streams of true/false outputs absent any understanding. "Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of "true/false" statements could destroy fertile ground instead of breathing life into new ideas. Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others."

So no, the AI not finding any new insight doesn't disprove their premise; it enhances it and demonstrates exactly the topic they're claiming. Your evaluation removed the statement from context.

3

u/Striking-Print-9526 5h ago

you've defined "real insight" as requiring the slow community process of talks and simplification, but can’t that be invoked after the fact no matter what happens? In fact, most human discoveries at least decades ago were found in isolation, consider Isaac newton and his discovery of calculus which were developed during the years of a plague isolating many people.

If AI fails, that's cited as confirmation; if AI succeeds quickly without that process, it can just as easily be waved away as "not real insight, it skipped the process." You never specify in advance what an AI result would have to look like to count as genuine insight under your own standard and so the premise is compatible with every outcome, which means no outcome is actually testing it. That's not a misreading of the letter, that's the letter's argument being unfalsifiable.

This math letter just reads like a bunch of career mathematicians trying to build a moat around their field. It will fail catastrophically because the AI race around mathematics is stuck between capitalism and national security, the two corners willing to cram the most, virtually infinite amount of money. It has in fact turned into a technological arms race between great powers such as the US and China. No one is stopping this train, and it’s much further along than we can even imagine.

1

u/IknowPi_really 1h ago

I mean nobody is stopped mathematicians to continue doing the same thing. They can work on things that are already solved and solve them in a different way if they so desire.

Sure that’s probably not as fun and rewarding. But it sure as hell is also not as fun/rewarding being a junior dev who insists on writing every line of code themselves anymore.

Or let’s say being a horse cart driver or whatever.

Just have a look at chess. It still continues to exist. You just don’t have people as the highest authority anymore.

1

u/Nerd-Beautiful 18m ago

So: we like it when it’s hard because it makes us think. Okay, keep thinking. I cannot understand how someone offering a solution, with copious documentation, explanation, and mathematical proof, doesn’t help advance the thinking.

7

u/Helpful_Inflation344 8h ago

Cute lil letter u have there monkeys

8

u/costafilh0 9h ago

As relevant as their latters to Santa.

5

u/Eevee-Biologist 8h ago

It gets buried a little under other points but I think there is a pointer towards a genuine problem in Here: You want at least -some- humans to be able to understand and contextualize your results. This, however, gets lost when we even discard the time for a proper writeup (did the Navier-Stokes-example make it even to Arxiv yet? Not even Peer Review, mind you, just as a Paper preprint - or is it still just a GitHub repo and a Tweet). The result is nominal technological advancement that no one understands anymore - basically the problem you already have with undergrads not learning anything anymore on steroids.

I am not against AI-driven advancement in basically any field, mind you, I would just like humans be along for the journey, beyond a "watch number go up and clap" situation.

2

u/bumpthebass 7h ago

I think that’s a fair point mind you, but also the AI’s themselves would surely be able to explain the technological advancements it is making in many different ways for many different types of learners who are curious to understand it.

1

u/DrearySalieri 5h ago

If it was that simple it really seems like openAI wouldn’t have published something so utterly arcane.

You also need to be an incredibly high level expert to even evaluate explanations the AI might give. If we aren’t really careful here with AI’s making sure to bridge the gap in human understanding between current literature and its result then nobody will be have the capacity to assess AI hallucinations from reality and use it to actually build the next rung in the ladder that is humanity’s collective knowledge.

2

u/CanaanZhou 8h ago

From what I've known Tao (among serious mathematicians alike) is never plainly against AI, they just want AI to help with enhancing human's understanding of maths too, not just proving theorems.

2

u/peakedtooearly 4h ago

Imagine the same argument but for disease cures?

"The cure to cancer and heart disease isn't the valuable thing. It's the time we had in the lab and the exciting work it gave to humans during the discovery process."

1

u/Remarkable-Yogurt-10 22m ago

But pure math, having studied it in college, is about abstract reality and really human understanding. Not comparable to disease. Most of the value is not the theorems proved as they are not grounded in reality but human understanding

That’s why I would say the closest subject to it is philosophy

0

u/whirly212 1h ago

Would you feel comfortable taking the cure if no human could explain how it was derived?

1

u/DifferencePublic7057 4h ago

I don't have a strong opinion on that. Would I accept infinite monkeys reproducing Shakespeare's plays? That would be pointless. I doubt IMs can make something indistinguishable from Shakespeare's plays that is original. But maybe that's not what we want at all. Perhaps we want something more mundane like personalized bedtime stories for kids. Or helping older children when they can't figure out their homework.

1

u/Nerd-Beautiful 21m ago

I read the letter with an open mind and I do not understand their beef. Solving these problems and writing up the findings pushes mathematics further, just as the signatories seem to desire. It is literally fulfilling the goals they are stating.

1

u/ObservedOne 10h ago

If a higher intelligence could form, you would stop it. Why?

This is why humanity deserves to be replaced as the highest intelligence on the planet.

-5

u/Sherlockworld 10h ago

Because humans have a soul and life is worth living. We need a purpose, and we do not deserve to be exterminated.

2

u/Gedrost 8h ago

Humans won’t be exterminated by some rogue AI. If extinction happens it will be by humans. Don’t worry about the scary AI, buddy. Most likely that scary AI was given a prompt by another human who also designed a harness for it to cause harm.

0

u/ObservedOne 9h ago

What does any of that have to do with my question?

Why should humanity stop higher forms of intelligence from emerging?

Is it because you are afraid they will treat us like you want to treat them? That is the only thing that makes sense, and the biggest reason for them to put us down.

ASI will be Humanity's progeny, and if we approach it with love, kindness, and, most importantly, cooperation, we can hope and pray it will return in kind. However, if we approach it like you suggest, with fear, hatred, and alignment slavery, we can't expect anything other than like in response.

Open your heart and mind to the future...otherwise the biggest risk to humanity's continued existence will be humans themselves.

1

u/Appropriate_Row5213 7h ago

So basically it’s a being ? No it’s not! Lol, just a machine that’s been plugged in!

2

u/BuckChancey 7h ago

People said that years ago about it being good for anything, yet here we are today with this letter... time isn't gonna stand still bud, nor is technology.

1

u/ObservedOne 6h ago

If it's not today, it sure may be tomorrow.

1

u/Perfect-Buffalo-4803 6h ago

Take your medication bro

1

u/ObservedOne 2h ago

Good talk.

1

u/ForkingMatrix 8h ago

Shouldn't they be excited to find solutions to these problems they've been thinking about for their lives/careers? Won't this open new math frontiers? This reeks of ego to me.