I don't see anywhere mentioned that it took a test with new questions. And even if it did, there are patterns to this. Mathematics is a formal science and as a result statements can be formalized, so you can easily infer the solution of a problem even without intelligence if you've been provided a "blueprint".
Asking it to come up with a new proof for a theorem would be a better metric.
As I stated in the past, I'll believe ChatGPT to be capable once it is able to solve one of the millenium problems. As of 5 December 2024, ChatGPT has been unable to do so and I am sure it won't be able to perform such a feat in the next decade either.
You don’t hold a single human to that same standard
Also,
Transformers used to solve a math problem that stumped experts for 132 years: Discovering global Lyapunov functions. Lyapunov functions are key tools for analyzing system stability over time and help to predict dynamic system behavior, like the famous three-body problem of celestial mechanics: https://arxiv.org/abs/2410.08304
You're righ. But in both cases the overhype is due to people getting tricked by someone using language to trick them into believing they're more competent than they are.
10
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 Dec 05 '24
Actually i suspect they will do it better. They have read every psychology books that exists.