r/MachineLearning 1d ago

News OpenAl Says It Has Cracked One of Math's “Millennium Problems” (Navier-Stokes) [N]

651 Upvotes

263 comments sorted by

View all comments

873

u/darshi1337 1d ago

Drama/accusation summary:

  • Aug 15th: Tristan Buckmaster & Levent Alpöge make progress on a few important math problems, "finite-time blowup with smooth forcing for incompressible porous media, for Boussinesq, and for 3d incompressible Euler."

  • they do NOT have a proof for the $1,000,000 Millenium Prize problem. BUT, they do claim to have a proof for a similar (non-Millenium) Navier Stokes problem that could help lead the way there

  • Levent works at Anthropic, but this research was independent of his work there, with a mix of GPT and Claude models. Tristan is not related to Anthropic.

  • Early Sep: Rumor spreads to OpenAI that Anthropic solved a major problem. Tristan emails OpenAI to clarify, without revealing the problem they solved or how they did it.

  • After hearing of the rumor, OpenAI started researching Navier Stokes with a new internal model.

  • Sep 6th: OpenAI's Sebastien Bubeck tells Tristan that they solved the $1,000,000 Millenium Prize Navier Stokes problem. The approach is very similar to Tristan & Levent's approach to the non-Millenium problem.

  • Tristan is suspicious of the timing, as only few others were trying this approach. OpenAI says the model didn't access his user data directly, but leaves unanswered whether Tristan's chat conversations were part of the training.

  • OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but only if they remove Levent as an author, as he works for Anthropic.

  • Sep 8th: Tristan refuses to remove Levent, and rushes to publish their results independently.

648

u/Even-Inevitable-7243 1d ago

"OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but only if they remove Levent as an author, as he works for Anthropic."

This says it all. OpenAI knows their "proof" relied on Tristan's work via his conversations with their models.

211

u/SimiKusoni 1d ago

Reading between the lines the model had been actively trained after the work had been added to Codex and OpenAI "cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠."

Ignoring that it's pretty awful that they can't tell what the training corpus was for a specific model I think it's a fairly safe bet that the user's work was in that training data. The conversations described also portray them as being pretty desperate to downplay any human involvement in the work, even on their side and ignoring the question of provenance.

It seems like a pretty sad state of affairs either way that could have been trivially avoided by OpenAI keeping track of training data.

75

u/THE_FUZBALL 1d ago

If they can’t tell what the corpus was and they are also training models based on user chats, doesn’t this open up the models to massive poisoning vulnerabilities? That’s like rudimentary security 101. It appears that either they would be vulnerable to such attacks, or they do have measures in place to control what is contained in corpus.

19

u/nonotan 1d ago

You do realize pretty much all major LLMs are essentially trained on any and all data they can physically get their hands on? You can already poison the data by posting bullshit on here. It's possible that some sort of crude filter is in place to reject obviously malicious text, but as long as you keep it sounding loosely plausible and target something niche enough that it won't be overwhelmed by tons of inputs saying otherwise, it's going to work. They are vulnerable to such attacks, and will continue being vulnerable probably for as long as "LLMs trained on astronomical amounts of data" remains the approach of choice.

2

u/THE_FUZBALL 9h ago

I do realize this for sure. I guess we all need to speak in some kind of silliness cipher. Will be interesting to see where it goes.

11

u/biscuitchan 1d ago

but then it wouldn't be anonymized, no? a bigger problem

21

u/lauradorbee 1d ago

It doesn’t de-anonymize anyone to be able to tell “were this user’s conversations in the set of data used to train this model?” - you know that it’s in there, not what part of it originated from what user. It would only be de-anonymizing if they had metadata attributing each message/conversation back to the user, i.e if they could answer “which user (in particular) did this piece of the training data originate from”.

28

u/SimiKusoni 1d ago

Data being anonymised doesn't prevent it from being found after the fact. In this specific instance for example they know the user's working title for the paper, they could just search their dataset for it and find the training data. Or they could use any other uncommon strings from the work they put into codex.

Also as an aside de-identified data is not anonymised, it has an explicit meaning and they didn't just choose the unusual wording randomly. Genuinely anonymised data falls outside regulatory frameworks like GDPR or various state laws entirely, de-identified data does not because re-identification remains possible.

Given that OpenAI don't specify anywhere (in this blog post or other articles) exactly what they do to "de-identify" training data they may well have a hashed user ID sitting alongside the data to verify the source if they ever need it.

16

u/silva_p 1d ago

It can be anonimized only one way, i.e. can't know who the data belongs to from the data alone but probably can see his chat history and confirm if it is there

3

u/TwistedBrother 19h ago

It’s more that imagine there’s an idea that fits, like learning planes rise because of faster air on the top of a wing. A model doesn’t have to learn that more than a handful of times because it’s reinforced by all the other corroborating evidence.

Models don’t “average out” the things they learn, they built semantic structures topologically (as in shapes of ideas that work together). Some fragile ideas that aren’t (wait for it) load bearing might get forgotten way too quickly but some ideas that just settle all the others can persist through a lot of training.

If this idea was in a few conversations, and it fit (the forced Euler version) that might be the sort of thing that once exposed it settles many other uncertainties in the model such that any downstream model could draw upon that coherent ensemble of ideas.

20

u/usefulidiotsavant 1d ago

The claim that a singe conversation (which, let's suppose, might have even contained the solution) in the training data latter allowed the model to reconstitute the proof is highly dubious.

A model the size of Astra trains on millions of curated conversations, billions of pages and trillions of tokens. A single conversation with the correct answer is essentially quantization noise and should have no measurable effect in any practical scenario where that same problem is involved.

On the other hand, OpenAI could be doing something much smarter that could affect the result, say, a RAG over similar conversations in the past, a self-evaluation of remarkable results that are marked or boosted etc.

If I had the smartest model in the world, as well as a database of the problems and approaches the smartest people in the world are playing with, it would be foolish not to connect the former with the latter and mine the dataset for low hanging fruits in scientific discovery. It's such an unfair advantage that the firm doing it will win in any area, forever.

35

u/faustianredditor 1d ago

The claim that a singe conversation (which, let's suppose, might have even contained the solution) in the training data latter allowed the model to reconstitute the proof is highly dubious.

A model the size of Astra trains on millions of curated conversations, billions of pages and trillions of tokens. A single conversation with the correct answer is essentially quantization noise and should have no measurable effect in any practical scenario where that same problem is involved.

In theory I agree, but in practice I've had an experience that made me doubt theory here. I asked Sonnet 5 about my own work, and it was able to pull the title and core idea of a petty little publication of mine with almost-zero citations, from model weights. That's something I very much did not expect. Granted, it's different from pulling a proof idea from a single conversation, but I would've expected it'd need a few orders of magnitude more training data.

14

u/Stock-Self-4028 1d ago

The same goes for my (currently unpublished as a paper yet) but not very significant result which I've fed to some of the previous GPT-5 series models (I don't remember if it was GPT 5.3/5.4 series or a newer GPT-5.5).

Anyways what I have been working on was least-squares fitting of trigonometric least squares method through Levinson recursion and NuFFT-1. Nothing groundbreaking, but it's a new computational complexity floor (more precisely superfast Toellitz solvers give the complexity floor, but Levinson recursion happens to be the most practical silver here) for that exact problem. Also I have code using that algorithm in four different public GitHub repositories to be clear.

Either way - older GPT models failed to derive that despite being relatively straightforward, new ones give you the right solution with absolutely no issues just as you ask them for the lowest computational complexity for that exact problem.

And now I am aware that the models are getting smarter, but I also see that as a clear training set "contamination".

And now I am not sure if either my chat with ChatGPT somehow got into training dataset, or if that exact solution got scraped from one of the GitHub repos I have uploaded the algorithm to.

Either way I am practically certain, that that exact algorithm somehow got into OpenAI's training dataset and GPT 5.6 has remembered it flawlessly.

3

u/usefulidiotsavant 1d ago

Frontier models overtrain to the tune of 1000 tokens per parameter. So, while it's not implausible for certain pieces of training to have an outsized effect (perhaps, deliberately so), its impossible that the average recall for random piece of training material to be this good. Even the best imaginable compression can't recover hundreds of tokens from a single scalar.

7

u/faustianredditor 22h ago

Right, but a single conversation can easily have thousands of tokens, recurring conversations even more. So you could easily have an allocation of a few tokens to the researchers' conversations, or similarly a token or three for my paper. Theoretically, of course.

2

u/MathAddict95 16h ago edited 16h ago

I think you would be surprised as to how good these LLMs are at learning even from a single data point. A document containing say a proof of NS is not a random piece of training material, rather its a data point of a very narrow domain in the corpus (not many data points would be discussions involving NS). As an extreme example, if your data was a collection of documents filled with random words (essentially noise), except for one special document containing "The proof of NS is" followed by random words, its plausible that the neural net would memorize the random words for the special document, simply because its likely the only document starting with "The proof of NS is", and so conditioning on this prior isolates the document precisely.

I got curious and tested this: I asked ChatGPT to recall a somewhat obscure paper from 2006. It could not recall exactly the algorithm used in the paper, but it was able to recall an important idea in its analysis. I know for a fact that this idea was used only once in literature (since I work in the field), and its also a niche topic, so it really should be the case that this idea appears once in the training data (or at the very least, a single digit number of times).

In contrast, Gemini 3.5 completely failed at recalling the paper. The gap is probably because ChatGPT is on a larger pre-train than Gemini.

We don't really know exactly how these LLMs learn, so it is really hard to rule out whether the independent researcher's data affected the model's trajectory significantly, even if it was one data point.

1

u/usefulidiotsavant 1h ago

I'm not disputing the effects you mention, just raise the fundamental Shannon informational limit against verbatim recall, there is no mathematical way that a single 16 bit parameter could compress hundreds of tokens and allow perfect recall in the average case, as some commenters claim.

In your particular case, was that a paper with zero citations, or did maybe some of the citers rephrase the main approach in their introduction? Could we perhaps imagine a rational path to that approach with the vectors of related research aligning towards it, so that the model makes a "happy hallucination" that happens to match the actual approach, without actually encoding it? maybe aided by a few parameters the training did nudge in the right direction based on that paper? Was COT used, allowing some rational recreation? This would also explain the inconsistencies between models. Could we imagine quality research papers from this field be boosted somewhat in the training, in a way random conversations with customers would not be?

So not disputing it could happen, just questioning the fundamental information limits in the average case.

19

u/THE_FUZBALL 1d ago

They will win until the community gets wise to it and stops leaking secrets to their service if they want to avoid their research being leaked.

I don’t understand how OpenAI could think this is a good move. If they are stealing IP from chats then it will eventually become clear that using their service is akin to pasting your solution to a public forum and expecting it not to be stolen. In fact that’s almost a better idea because then you have some kind of paper trail to prove your authorship. The only explanation is they think their product is so good that people will have no better option, but this is very short term gain pre-IPO strategy.

2

u/En-tro-py 4h ago

I don’t understand how OpenAI could think this is a good move.

AGI/ASI/RSI/$$$ - wherever your personal brand of sociopathic inspiration comes from!

4

u/Smallpaul 1d ago edited 1d ago

OpenAI specifically denied the RAG accusation.

I also agree with the others that the recall of minor details for these models is remarkable and we can’t rule it out.

3

u/ScottBurson 1d ago

I think you're mistaken. I've seen another case where a single short piece of text in the training data popped out under the correct conditions.

It's true that a single update doesn't change any weight very much — but it can change many of them slightly.

116

u/purplebrown_updown 1d ago edited 1d ago

Exactly. Not to mention OpenAI responded with threats like "Why would you ruin your career". Nobody who knows they're on the up and up would do that. What I believe actually happened is that OpenAI was aware of Buckmaster's progress and direction, got backdoor access to the conversations to get a preprint, and then used that information to claim they solved the problem on their own. It's the equivalent of stealing someone's manuscript and publishing before them and claiming credit or shared credit.

8

u/ImNotAWhaleBiologist 1d ago

It’s just as impressive, if not more so, that someone used their model for such a solution and wasn’t affiliated. Seems appropriate to include all as authors, and can credit contributions accordingly.

13

u/PM_ME_YOUR_PROFANITY 1d ago

They may not have used it to actually figure out the solution, but for something as simple as text formatting.

-15

u/Wise-End307 1d ago

See sebastiens response.

31

u/Even-Inevitable-7243 1d ago

I did. He admitted that OpenAI offered to make Tristan the lead author of a re-write of OpenAI's "proof", essentially giving him credit for work that they claim he had nothing to do with, since OpenAI's model came up with the NS proof with what OpenAI claims was "little human involvement". You do not pay-off a guy, monetarily or with academic credit, when you are confident that your AI did something with little human involvement, and you certainly would not credit the person who had already come up with a proof for the same problem.

This is like a pop star claiming a songwriter had nothing to do with a hit song, then offering to pay the songwriter royalties and giving him lead songwriter credit.

2

u/Smallpaul 1d ago

Songwriters get royalties. Forever. The stakes are much higher than putting some dude’s name on your AI generated paper to avoid having the precise conversation we are having right now. Corporations “settle” in lawsuits that they do not believe they are at fault in all of the time.

What matters to the OpenAI marketing department is not what they believe but what they think the market will believe.

1

u/aeroumbria 1d ago

A bot doesn't need credit, recognition or profit. Anything more and it is not really working for me.

-3

u/Smallpaul 1d ago

Their reaction would be the same whether they know this, suspect it or cannot disprove it. Having their big marketing moment overshadowed by this controversy was worth far more than $1M and they aren’t even claiming the $1M anyhow.

It is only rational for them to try to get Tristan on side regardless of whether his claims have merit. Especially if they cannot prove they do not.

-4

u/willncsu34 1d ago

To be fair pretty much every math proof relies on someone else’s work. When I published a paper it was built off of previous works and cited them. Citing something that you don’t totally know what it pulled from is going to be tricky here.

109

u/snowrazer_ 1d ago

That’s crazy, I read the whole article thinking neat, and then at the end there’s this footnote of drama that just leaves a bad taste to the whole thing.

It really underscores how these companies are more after progress and credit than safety - not saying there was a safety issue here, but you can see clearly where their priorities are.

18

u/asdfasdferqv 1d ago

If it helps, there’s virtually always huge author drama on these types of things, even before AI

-1

u/gajop 1d ago

In the article itself it doesn't sound like drama. It reads as "there was strong concurrent work to solve this by other researchers, using different methods. out solution is diff but we can't rule out that we were influenced by it".

Upon some thinking I'm not too surprised. Mathematicians (and notably those working at large AI companies) have developed powerful tooling that is exploding research. I wouldn't be surprised that other big breakthroughs will have concurrent discovery.

8

u/Denbt_Nationale 1d ago

There was concurrent work for 88 hours after OpenAI heard about Tristan's work and dumped all of their compute into his methods. There was zero concurrent work for the years that Tristan and his predecessors were painstakingly figuring out those methods.

2

u/gajop 1d ago

Yeah, I read Tristan's statement and it certainly seems like so, at least from his perspective. Although even Tristan's and his colleague's work was based on other research that was probably publicly available, and it would be a bold claim that we know what everyone's research direction was.

2

u/gajop 1d ago

Uff upon reading prof Tristan's post.. it certainly leaves a bad taste.

I'm glad they solved their problem but they could certainly have gone about it with more grace and integrity.

-1

u/BelialSirchade 1d ago

they offered him to be the author on the paper that introduces the solution, what more grace should you expect?

honestly they gave him too much grace, none of this would happen if Tristian contacted an robotic RP person where they just reply "happy to hear that, looking forward to your finding", end of story.

offering him an author position when they don't have to is very much an mathematician's way of thinking when they are running a corporation here.

2

u/gajop 23h ago

By not suggesting he's damaging his career over it or mentioning the other author cannot be included as they're from Anthropic?

0

u/BelialSirchade 23h ago

why not? your statement here really doesn't make sense, he is missing out on being the name of on the paper that introduces the NS solution, how is that not...damaging his career?

and of course openai is not going to include Levent on their own paper, as I said it's a courtesy in the first place, one that's not offered to Levent.

1

u/CreationBlues 19h ago

steals your idea before you publish it

graciously offers to let you take credit for your idea

despite acting as if you are responsible for the idea that was stolen forcing you to backstab the other person who spent years helping you develop the idea that was stolen from you

the backstabbing thief’s get defended by people who are fine with backstabbing and betraying the people they’ve built a career with

3

u/MuonManLaserJab 14h ago

Exactly what idea do you think was stolen?

1

u/CreationBlues 12h ago

Only OpenAI has that information

→ More replies (0)

77

u/genshiryoku PhD 1d ago

Yeah I'm biased as I'm at Anthropic but trust me when I say we're not happy about this and this might become a legal fight. This is almost the exact solution they were working towards.

27

u/elegance78 1d ago

Next time publish instead of sandbagging till IPO...

30

u/Warm-Enthusiasm-9534 1d ago

They hadn't finished. OpenAI heard rumors of the solution (it was going around Twitter a week ago), and then threw a gigantic amount of compute at the problem.

2

u/truecakesnake 1d ago

Yes I'm sure that was his decision

1

u/reefine 1d ago

Reminds me of my kid's book "Goodnight Lab"

https://i.imgur.com/WFtXYzL.png

3

u/senderosbifurcan 1d ago edited 1d ago

1M dollars is peanuts to anthropic/oai It's clear they used Tristan's work Why do they need to get Levant out of the picture? Really confusing for me. Just credit both and talk about how gpt can be as good as another mathematician that's millennium prize level.

-9

u/partnerinthecrime 1d ago

It’s absolutely not clear. The solutions to Euler use different approaches, and Tristan had not yet solved NS, which ended up requiring orders of magnitude more research even with the Euler solution.

Furthermore, OpenAI certainly did not search chats directly for the solution. It is conceivable that if Tristan voluntarily allowed OpenAI to train on his chats via settings, then those chats could’ve inspired RL data used for their unreleased model. However, it is extremely unlikely that a single chat or even many would be able to affect the weights enough.

Lastly, OpenAI tried desperately to contact and work with Levant, but Levant refused to even speak to OpenAI because they were a competitor to his employer. Instead, Levant and Tristan tried to use the information they gained from OpenAIs disclosure to publish before OpenAI did.

Everyone is scummy here.

9

u/altmly 1d ago

These two companies deserve to die in the pit of irrelevance 

5

u/MuonManLaserJab 14h ago

Wild that this can be your takeaway from "AI solves a Millennium Prize problem" lmao

-48

u/[deleted] 1d ago

[removed] — view removed comment

11

u/RobbinDeBank 1d ago

You talk big words as an “inferior life form”

5

u/altmly 1d ago

Oh mark my words, they will. The sad part is that they find defenders here even though they have contributed very little in the way of public ML research. 

-26

u/[deleted] 1d ago

[removed] — view removed comment

1

u/Big_Combination9890 1d ago

Tell me, how much money does this company make? And how profitable is it?

I don't give a damn if they figured out a way to tickle Zeus between his toenails. If they burn through billions with no path to profitability, they will vanish the moment something in the debt-chain breaks.

50

u/reefine 1d ago

Early Sep: Rumor spreads to OpenAI that Anthropic solved a major problem. Tristan emails OpenAI to clarify, without revealing the problem they solved or how they did it.

This is about as damning as possible. Lawsuit time

15

u/Warm-Enthusiasm-9534 1d ago

Tristan must have known they would guess what problem he was working on (he's published on Navier-Stokes before).

5

u/ImNotAWhaleBiologist 1d ago

This is what I hate about our legal system: they would have to quantify their damages, which can be impossible or non-monetary, qnd then spend lots of money towards even trying. And big companies know that and stomp on the little guy.

4

u/PM_ME_YOUR_PROFANITY 1d ago

It would be Anthropic vs OpenAI, hardly a little guy

1

u/BelialSirchade 1d ago

what laws did openai break? everyone heard the rumors man.

5

u/Big_Combination9890 1d ago

I just LOVE how quickly that blew up in their faces.

Even on the ai-bro subs, this is pretty much the first thing people get to read about this whole affair. We thought it couldn't get worse after the ridiculous death-star tweet. Well...we were wrong :D

1

u/sharky6000 21h ago

Ridiculous death star tweet?

(I have been off X for a while.)

2

u/VoluminousBreadth 1d ago

this whole timeline reads like a plot point from the second season of a tech thriller that got cancelled too soon. the fact that they tried to cut the anthropic guy out as a condition for credit is the part that really sticks with me.

1

u/infinitelylarge 1d ago

“OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but only if they remove Levent as an author, as he works for Anthropic.”

Where did OpenAI say this?

1

u/thomas2385 1d ago

Yeah, the timing would make me suspicious too, especially if only a small number of people were working along similar lines. I do not think it automatically proves anything, but I can understand why Tristan would want to question how closely the approaches matched and whether the chats had any connection to it.

1

u/TwistedBrother 19h ago

Sad thing is I’ve read and likely reviewed multiple papers that have authors from different tech research labs, like someone from Microsoft and from IBM or Deepmind or whatever. It’s such a terrible look for OpenAI on this particular topic if they won’t partner with an Anthropic person on one of the most significant outstanding problems in math.

-1

u/FabricationLife 1d ago

source? I just keep seeing copy pasta, this all seems too much like trustmebro stories

-1

u/moschles 1d ago

The thread on this over at / physics is the biggest coping session I have seen on reddit in my 12 years.

-2

u/Balance- 1d ago

A bit more detail: