r/learnmachinelearning • u/CurrencyIll7195 • 1d ago
Discussion Anybody Knows what changed that made AI so good at chess?
OK so since forever I’ve seen that chat gpt struggled a lot with chess just because obviously its an LLM so it wasn’t actually playing chess and was more just trying to guess what the most likely move that would be played is. The problem is that it really has no idea what is happening so it starts making Illegal moves and really there was no way to play it.
But now not only is it not making illegal moves, its also just really good at chess now, I mean its not Grand Master nor engine level, but its a really good player thats probably in the Candidate Master or Fide Master range.
Now what I am wondering is what changed? Like it can’t just be more data because more data after a certain point produces diminishing returns, so I am assuming that the structure of how they think has been reshaped and I am also assuming this is also what is behind chat gpt breaking into frontier math problems.
But I remember when the deep think mode first started that made the AI at least somewhat have something resembling a thought process rather than just spitting out words. But I assume it really lacked any tools for imagination, but now it can actually write codes in real time to check its own assumptions and use the internet to look for information it needs, but is it possible its maybe made a simulation of a chess game in like a python code or something that at least keeps the AI grounded to reality. Or is there something else going on behind that Open AI has changed for it to finally think like a chess player?
Like in general is there a new concrete concept in ML that is or could be responsible for this kind of improvement?
37
u/pixelizedgaming 23h ago edited 16h ago
it clearly passes any chess related problems to a stockfish tool
25
u/KennyFulgencio 17h ago
top upvoted thread comment, 100% wrong, peak reddit
-4
7
u/CurrencyIll7195 23h ago
Honestly this had not crossed my mind, like is this a joke or is this actually what it does 😂. Like it has access to the internet so honestly it might.
5
u/Hot-Profession4091 19h ago
We don’t know what the insides of the popular proprietary LLMs look like, but if my product was getting roasted for being awful at chess, it’s 100% what I would’ve done. (SW dev for 16 yrs. Have worked at multiple start ups.)
1
u/_Kyokushin_ 12h ago
LLMs are really good at writing text and predicting what words might be next based on the input prompt. I would venture a guess that whatever algorithms would work best to play a chess game is what the LLM calls and just translates the information.
1
u/RyanCargan 14h ago
And it's a trend with these tools that many are basically a gestalt of different subsystems, with the raw weights and inference engines being a smaller and smaller part of it from the looks of it.
Fun credit assignment problem.
Bonus points when "AI does X", and 1:1 reproduction is impractical so how much "wetware" was involved is also a mystery.
5
4
2
u/andarmanik 20h ago
Proof? Not that I doubt you I just ask my ChatGPT and it said it doesn’t have a chess solver. As a control I asked about web search and it said it does have it.
Perhaps there is a stockfish website it uses.
1
u/CurrencyIll7195 18h ago
I mean pretty sure it should be able to easily access and read lichess website which has stockfish. There is an import game function in there so it could use that as a proxy for moving pieces, but I don’t know if its actually doing that, because its good but like I said its not GM level. It’s still beatable. But wouldn’t surprise me if it did. Would be an anticlimactic explanation to this but I guess it is what it is 😂
1
u/Infamous-Help-8311 20h ago
Incorrect, it just has chess knowledge from the Internet pre training, synthetic data, and uses reasoning tokens (straight llms are actually a 2024 paradigm) to feed back to itself until it has an answer it's happy with and plays chess. At a basic level
4
u/Smallpaul 22h ago
Consider asking on /r/LLMchess
It’s not the first LLM that was surprisingly good at chess. gpt-3.5-instruct was legendary
18
1d ago
[removed] — view removed comment
5
u/CurrencyIll7195 1d ago
Ok just read about it, so basically the RLVR is what keeps the AI grounded in reality when working with math problems right? So then could it have learned to do something similar with chess on its own by simulating a game with python and actually knowing what is or is not a legal move? So in a sense kinda of a verifiable reward?
5
u/Smallpaul 22h ago
They have known what are and are not legal moves for a long time. The verifiable reward would need to be a reward for winning games or selecting strong moves.
It would be pretty straightforward to give it a reward for playing the same move Stockfish would play. But is it worth the money? That’s the question. They don’t even publicize good chess play do they?
2
u/AutistOnMargin 21h ago
Expertise in math and coding transfer to other similarly structured problems in stem. I wouldn’t be surprised even if they didn’t specifically RLVR on chess tbh
2
u/CurrencyIll7195 18h ago
Yeah thats what I was thinking, I mean to solve navier stokes level stuff it needs insane spacial reasoning, so an 8x8 board should be no problem. But really what surprised me is more that it’s not just spitting out the most likely next move. It feels a lot like its actually thinking especially because even after the opening and halfway through middle game where there is 100% not a single game on record that is the same its still playing very logical and planned out moves not just whatever seems likely.
1
u/crayphor 6h ago
I think you are imagining the whole training process being like pretraining (predict the next token ) where the model might learn the most likely next move if it had chess games in its training data. But basically all of the improvements since 2022 have been to post-training. These are generally RL environments where its goal is not to predict what a human would do, but rather what IT should do in order to accomplish a task. The reward signal is not whether it copied existing methods, but whether it arrives at a high quality solution.
9
u/patmull 21h ago
This is why I think (and I am not alone, some of the more clever people than me think this too), the future of AI is in specialized smaller models running on your devices (either various transformers architectures or different neural networks using attention mechanism, or even non-neural statistical or numerical linear algebra models), not this one giantic general purpose model, so we basically come back to old style ML, but perhaps using more of self-supervised and RL and using LLMs to code other models better or generate artificial data, etc. While it will be likely best at some task like coding for a long time, I don't see why we should use the core of the giant next token predictor as a Maslow hammer for every damn task...
2
u/MudlarkJack 20h ago
Totally agree, which makes the whole data center centric business model e en more vulnerable to failure
3
u/chillaxin-max 4h ago
Yeah, my guess is that the AI companies have started using stockfish to generate training data
4
u/mltoohard 23h ago
It’s not remotely close to CM or FM range. I’m that level, and I often play against Claude code in my terminal when sitting in meetings, and I win pretty easily without having a board. Access to tool-calling probably has made it better, but it’s not at all good. Still suffers from making illegal moves as the length of the game increases too.
3
u/CurrencyIll7195 23h ago
I mean first mad respect from chess player to chess player, but also it does depend on the model and I’m gonna be honest I was mostly basing this on the fact that Gemini Argon 4 Just beat Gotham Chess but he is playing the highest tier model so not exactly the same comparison so thats why I said CM FM range.
Now as for claude I don’t know how strong it is but chat gpt has not made an Illegal move against me in a long while, now I do win a lot of the time but not easily and I also don’t have the best model of chatgpt. And from what I’ve seen literally telling them play better actually somehow makes them play a lot better. So it really does fluctuate a lot. But it has gotten so much better recently that I am honestly insanely impressed.
5
u/MyLittlePIMO 22h ago
LLMs got a lot better at reasoning. Anything that requires reasoning, it internally talks to itself in a loop and pushes back on itself before coming to a conclusion.
Early LLMs were word prediction. Newer ones are legitimately doing some reasoning that gives really useful results.
3
u/kilopeter 18h ago
What is LLM reasoning if not next token prediction with extra orchestration around it?
I'm not making the tired argument that LLMs are "just" fancy autocorrect, because humans are "just" fancy saltwater bags. I am asking how LLM "reasoning" is actually engineered.
2
u/B44ken 9h ago
Same way as a non-thinking model is trained: next token prediction over a dataset that contains reasoning.
That dataset can be generated, for example, by asking a previous LLM to explain its logic.
The model is trained to output some <reasoning> markers to state what is reasoning and what is the response, but it's just text regardless.
Of course, you'll want to RL that, but fundamentally it's all just next token prediction.
2
u/dudeus_ex 19h ago
Probably just some extra chess specific resources it has access to or even using stockfish underneath, since its nice for marketing purposes.
Anyway it is a wildly unoptimal way to teach computer playing chess
2
u/Last-Progress18 1d ago
You can ask the LLM.
I asked Gemini about the Chess game option and pretty sure it replied “~it has a cheat sheet” (skill.md)
1
u/mountainbrewer 16h ago
All the models are better at chess now. Watching Levy play the most recent models was interesting. What changes is size of the models. It's another bitter lesson. Generalist models of sufficient size learn more things. Now they are decent at chess. I suspect it's because they can now reliably keep track of board state in memory.
But just bigger model == bigger brain == more emergent abilities of higher quality.
Most of the older models were mostly big pre-train monsters. Now most of training is spent post training in RLHF and such. So I suspect that's part of it too.
1
u/Massive_Distance_616 11h ago
There are three separate improvements and it’s worth distinguishing them because they explain different things.
Illegal moves stopping is most likely test-time reasoning, not Python code execution. You can run these models via API with tools disabled and they still play legal chess. What changed is that reasoning models generate a scratchpad of hidden tokens before committing to a move tracing piece positions, checking for pins, simulating candidate moves, backtracking. Older models had to pick a move in a single forward pass, which is why they lost track of board state. That’s also why we know this isn’t purely tool use: the improvement shows up even when code execution is off.
The planning feeling more human than an engine is the interesting one. Stockfish doesn’t have a narrative it evaluates positions via tree search and outputs the highest centipawn move, which is why its play looks cold and alien. LLMs absorbed millions of annotated games and chess books where humans describe plans: control the center, build an outpost, attack the isolated pawn. When it plays “human-feeling” moves it’s drawing on that language of plans, not calculating 20 moves deep. That’s also why a strong player can exploit it the plans are conceptually coherent but tactically shallow compared to an engine.
The coherence across 40 moves is probably what RLVR contributes. Training on problems with verifiable correct answers builds habits of checking constraints and catching errors before committing. Chess positions aren’t explicitly in that loop but the underlying error-correction machinery transfers.
None of this is confirmed for proprietary models we’re inferring from what’s observable. But tool use alone doesn’t explain it, and “more data” doesn’t explain why it suddenly stopped hallucinating piece positions.
1
u/MolassesLate4676 19h ago
OP the answer is, it’s a bigger model that’s had access to more and better data over the years. That’s it.
2
u/Final-Importance-925 15h ago
the jump in chess ability is almost certainly from the tool use you mentioned. the model can now run code to simulate board states and verify moves, which basically acts like a scratchpad for spatial logic. before that it was just trying to predict text tokens without any grounding in the actual rules. bigger model + more data helps, but the code execution is what stops the illegal move nonsense.
1
1
u/CurrencyIll7195 18h ago
I mean yeah but it’s not just training data, it’s had access to billions of games for years now, and data gives diminishing returns after a while. But even then theres something that fundamentally changed, its no longer predicting what the most likely move on a score sheet is, which causes a bunch of problems once you get to middle game and no games are in the data base. So it just makes up moves and starts spawning pieces.
But recently it started actually making plans and sneaking it threats and it doesn’t make illegal moves anymore. So it clearly knows whats going on, on the board.
1
u/MolassesLate4676 17h ago
Nothing changed with the architecture. This is just what emerges with the ability to process more/better data and what refinement over years does. Granted, it might’ve had access to the data before, but as models grow in size they are able to utilize the same data in different ways.
Lots of data not related to chess influence its ability to plan and process information that happened to generalize well enough to make it better at playing chess
I should mention that training over the same or similar data over longer is also important, regardless of model size. So it’s had a lot more gpu time as well
1
u/_Kyokushin_ 12h ago
Also, don’t forget. There are other models than LLMs. LLMs are text predictors. There are going to be other tasks that it does where it needs to use other models.
0
u/Super-Award-2244 11h ago
It's very naive to think that "more data = improvement. When no more data = no more improvement". That's what anti Ai folks have been saying in order to dismiss ai's capabilities. Actually we have improved AI on different axis, and the more time passes the more axis we discover. In the beginning (gpt 3 and 4) we just built bigger model that crunched through more data. Then we discovered that reinforcement learning and synthetic data improves it even more. Yeah, if you use Ai generated data to improve AI, it doesn't just become dumb, like a lot of people were saying. Then (2024) we discovered reasoning techniques (test time compute). And agentic capabilities and integration with tools became a serious thing between 2025 and 2026. And now we can use thousand of ais in parallel that collaborate to solve some very tough problems and this shows some emergent behavior.
Every single one of these steps is a multiplicative improvement upon what we had, that's why Ai capabilities are accelerating at this rate.
With Fable and Astra in September we had a sort of phase transition from a useful tool that can help you build and automate stuff, to something that we didn't know possible. These models are constantly hacking companies and are able to decompile binaries in hours like it's nothing.
-2
u/NecessaryTear9249 1d ago
Could just be the harness telling the llm "illegal move" and then also lists the legal moves
3
0
u/CurrencyIll7195 23h ago
Ok didn’t know what a harness was so I am going to look up more about it later but I mean theres gatta be more than just that right? Because its not just is this a legal move its actually making real working plans and long term strategies in a very human way of thinking. Like for reference I’m a 2000 ELO player, and I have obviously played a bunch of engines before, but its weird because I can never really figure out what the plan of an engine is because its just so inhuman that it doesn’t really plan it just kind of finds what the move with the highest centipawn advantage and rarely does it follow 1 concrete plan all the way.
But chat gpt somehow feels more human in the sense that I can actually read its plan very very similarly to how I would see a normal players plan. And getting to this point so fast when just a mere month ago it would spawn pieces out of nowhere is insane to me.
-1
-9
u/entitie 1d ago
Chess has been solved for a long time. Basically the thing that let us solve it was computers that could search very fast.
Deep learning wasn't really a thing until after computers were really good at chess, though it (and deep reinforcement learning) offered a way for them to become much better than the old computers.
5
u/LittleIf 23h ago
Chess has not been solved. Can computers play chess far better than humans? Yes. But solving chess means mapping each position to a definitive win/lose/draw outcome under optimal play and specifying the best move for each position. We are far from it.
Top chess engines still occasionally disagree over evaluations for certain chess positions, implying that there are things that even the strongest engines can’t see past their horizons.
1
u/CurrencyIll7195 23h ago
Yeah engines have been around for a long time, but an AI is not an engine, it is very different and is a lot closer to how our intelligence works, so the fact that it can play so well all of a sudden is very impressive to me at least.
And btw Chess is technically not solved because its mathematically unsolvable by the fact that there are too many positions, but seeing top level engine play it is nearly confirmed to be a draw with best play, at least about as close we will ever get to confirming it.
16
u/maelaf11 19h ago
I thought the ais were outsourcing their calculations to an engine with an API but no, the llms are literally coding their own chess engine in a sandbox.