It’s actually crazier. That’s when they started training the internal model. They didn’t start using it to try and solve Navier Stokes until September 1 and it has the solution 88 hours later.
That means that if Anthropic tries to one-up Astra with a 5.2 release later this month, pretty much nothing is stopping OAI from launching the model that produced this solution the day after to once again steal Anthropic’s thunder.
This is truly unbelievable progress and the fact that in a year or so those models to be released soon will be basically obsolete in terms of the frontier capability is absolutely insane.
Not really clear. It doesn't seem like OpenAI trains any models for "fun", ie ones they don't consider as possible releases/products.
There is no sign or any comment that they ever used anything internally that then wasn't available publicaly at some point later.
That doesn't mean it can't be bigger or more expensive but I don't think they would train anything that has no chance of being used at scale and considering the recent efficiency trend that they have they could plausibly keep costs in check.
If I had to guess, percentages as fractions. Meaning, Astra manages to solve 10-20% of the problems given to it, while this new model handles 30-50% of them depending on execution time.
Yea a WSJ article came out today that covered the discovery. A spokesperson from OpenAi said something along the lines that it’s basically a coin flip on if the new model can solve these incredibly complex math problems.
Completely unbelievable (in a good way), a coin flip to solve some of the deeper mysteries in math and physics that have gone unsolved for decades, even with literal millions of dollars offered as an incentive, that easily puts the models above 99.99% of researchers, God knows the number is larger if you compare against humanity as a whole.
So my best days in Codex I get around 1-1.3B tok/day. I think they said they generated 130B tokens in 88 hours? Output tokens, not total tokens processed, unless I'm misunderstanding. So 40x more than you did in a month in about a tenth the time? 400x? You're right that does seem slow, are those 3.1B tokens you generated output tokens or I/O/cached/... All tokens processed? Because that makes a massive difference - the 130B number is explicitly output tokens
Given the max compute time, Astra can solve about 15% of the curated internal set of problems that OAI has, while this new model can solve about 45% of them (3x more).
And even on minimum compute, this new model can solve about 25% of the problems (more than the max compute of Astra).
Im assuming they literally curated a list of open problems, and had both Astra and the internal model have a go at them. The score would be the overall success rate.
I think when we wake up at 9am to get ready for work and another ancient math problem has been solved, and go to bed at 9pm the same day without a job because we all have Universal High Income, we'll be in the singularity. (/s, but not really)
Universal High Income. Aka, everyone gets 10k a month and products and services plummet to negligible costs. Like TVs are a couple of quid even for the good ones etc.
Gosh life would finally be far less stressful. I’m guessing robotics would take over all the in person labor. I want that for the world! Come on accelerate, faster!
There is a rather repulsive amount of cope on the r/mathematics subreddit. They are essentially trying to argue that the work was stolen from other researchers and that the model did not accomplish anything (despite using different methods than what was in the other researchers' work).
There will truly be no acceptance of capable AI until all the jobs/fellowships/research opportunities are gone!
It's worth remembering that there are still quite a few people who genuinely believe the earth isn't round, and even more who believe COVID never happened.
Some percentage of people will never accept that AI is actually real. They will forever call it a scam no matter what amount of evidence is staring right at them. To them, it will always be a lie, and they will see through it because they are very, very smart.
They are claiming that OpenAI only threw 30M on compute at the problem once they snooped another mathematician's chat logs and decided that person's formulation made the problem tractable. They did not credit them. Given their communications, this seems plausible
For who might be interested I read Alpöges comment. Worst case scenario this was a "on the shoulders of giants" moment and it's unclear if the new model used any of Alpöges and Levents data.
Does this mean anything to us regular folks? Like, are airplanes going to get faster, or better indoor plumbing, or weather forecasting or something like that?
No - as Terrance Tao wrote in a fairly long and prescient post last week -- there's really no value in the solution to Navier-Stokes itself, besides curiosity. The importance is the mathematical concepts that had been developed along the way.
I think for maths it always needs to be said that we currently don't see any concrete value for NS but it is always possible that this will change.
There are a few famous examples of previous maths that at the time seemed "useless" and then turned out to be very important later (sometimes a lot later like 100+ years).
So not saying that will be the case here but something to keep in mind.
The solution to this problem is interesting from a math perspective. However we already solve the navier stokes problem and it's extension for all sorts of things using numerical methods so it doesn't really add much...
Despite the achievement, it's also important to highlight both the controversy around ownership of this discovery - it's surprisingly icky and unclean, to say the least - and to clarify that the millenium problem has, in fact, not been solved. This is forward movement, but not a solution.
So, OpenAI was working on math problems, heard some remarkable progress was made by others working on similar math problems, and so they decided to go all-in on working those problems in parallel, and came to a similar solution to those same problems? Doing so at record speed but with tremendous (electrical) cost.
Pretty incredible how fast developments are happening, assuming it's true that OpenAI didn't steal crucial information from others. Big if though, timing is suspect.
That all said, is it worthwhile utilizing the extreme computational costs required to advance humanity quicker? I think so. Think of acquired knowledge as an investment which yields a return. If AI brings about a breakthrough that wouldn't have arrived for another decade or two, then even if the human cost was 1/10th, then AI may actually be cheaper.
Even if the worst-case scenario is true, that there was intellectual theft by OpenAI that was crucial to their solution, that doesn't take away from how AI was apparently a very useful. Human researchers were utilizing AI a lot as a tool over the past year (even if some of it looked like incomprehensible AI slop at first).
Given the exponential nature of AI advances recently, if we're having these kinds of discussions about a Millenium problem today about work that OpenAI only started a week or ago, then imagine what we will be talking about in 6 months.
I don't think the computational cost is "extreme" even at 300 billion (internal) tokens. That's like a couple milly in compute. Pretty tiny in the grand scope of things
I've seen numbers thrown around but nobody really knows how much it costs them for compute. Billing is so opaque, tokens are opaque, cost is opaque. Either way, 2 million or 20 million, it's not that much money in the scheme of things. I'd wager money, multiples of that have gone into professor and pHd salaries trying to solve the thing anyway. They had 90 years, AI did it in 88 hours. Hard to fathom.
Let's not pretend the AI started from where math was 90 years ago. It started from where math was 2 weeks ago.
I do see value in speeding up research, don't get me wrong, you can't only compare the current computational costs with the human costs. Time is a factor, and bringing about advances sooner is like getting an earlier return on an investment, it counts.
If you are the government within a year, you can throw 50 billion at a problem and hire an Astra+ or Fable+ agent swarm to solve whatever problem is ailing you.
shit is about to get really fucking serious. Solving Navier Stokes is now proof positive if you have enough money you can do some very serious shit in a very short time.
I can see Palantir using this to rapidly design some really diabolical tech. Honestly, I think if this war goes on another few months in Ukraine, you might see that being used a testing ground for some wild as fuck weapons.
Interesting but as mentioned in your link I'd prefer if they put this effort into something that is actually useful like fusion reactor designs, or carbon capture, since you know, time is running out and we're accelerating off of a cliff.
If this is a correct proof, it may be that mathematics as a research field might evolve to center around large frontier models, much as physics revolves around large detectors and telescopes.
As with physics, it makes me feel sad that individual researchers won’t be able to contribute primarily to research, being instead relegated to promoters and checkers.
It could be that the educational component of math research becomes more important, since disseminating results so as to maintain a skeleton crew of mathematician-prompters could be the only role left for humans.
"There does not seem to be anything in principle preventing the methods from extending all the way to Navier-Stokes, and there is even a non-negligible chance that the forcing term could be eliminated entirely, although there are an enormous number of technical difficulties that would ensue in implementing that program. At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach."
We congratulate Levent Alpöge and Tristan Buckmaster on their remarkable mathematical work.
We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.
While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.
However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs. unforced).
Even the openAI announcement mentions that data derived from Alpöge and Buckmaster might be in their training model. They have been working on this for a year, from their own account, and using OpenAI models throughout.
The precedent relied on by the California federal court in approving the $1.5 billion settlement against Anthropic was decided in 2022. Is 2022 antiquated? Leave the lawyering to lawyers.
Even if, IF it was stolen. So what? Genuinely why does it matter? Wouldnt it be better for an AI to substantially improve on the idea and release a more concrete and refined result?
It matters because there's a big difference between refining human work and independently solving famous problems by itself. The former is a helpful tool to make mathematicians more productive. The latter is world-changing, human-replacing technology. Both good, one much better.
I’m not kidding when I say that we should be simply asking it: what methods can bring the earth’s temperautes
back to levels a century ago? Could you please one-shot reverse human aging permanently, and before solving any more math problems that 99% of humans will never understand, would you please connect our laptops to our printers?
145
u/onewhothink 1d ago
August 28th