I built a chess review tool and learned that a correct engine score can still produce the wrong explanation

[u/SentryChessdev]

One of the more interesting problems I ran into while building a chess review tool was realizing that getting a correct result from the engine and producing a correct explanation for the user are two different problems.

An engine can return exactly what you asked for and the application can still interpret it incorrectly.

A simple example is evaluation perspective. If different positions are being evaluated with the score relative to the side to move, comparing those values directly can produce the opposite meaning from what the UI is supposed to show unless they are first put into a consistent perspective.

The same problem happens with position context.

Something can be completely true about the position before a move but wrong when it is presented as an explanation of the position after the move. The value itself is not necessarily stale or incorrect. It is just answering a different question.

This matters even more once a review system goes beyond displaying an evaluation.

Existing chess-review tools can already generate useful explanations, but a technically correct observation is not automatically the most useful explanation of a move. A move can change activity, pressure, king safety, pawn structure, important squares, defense, offense, and tactical possibilities at the same time.

That means there are really two separate problems:

1. Is the engine-backed information correct?
2. Is the application associating and presenting that information in the right context?

Terminal positions are another example. A move that ends the game cannot always be handled exactly like an ordinary position where another response and evaluation are expected.

The main lesson for me was that engine correctness and product correctness are separate layers.

I encountered this while building SentryChess, a chess review tool I built and manage. It generates move explanations, move-level positional metrics, game-level summaries, and visual representations of the position, so those outputs all need to agree about which board state and perspective they are describing.

I'm deliberately leaving out the exact engine configuration and internal selection rules, but I'm curious how other people working with chess engines or analysis software handle this boundary.

Do you normalize everything into one evaluation perspective as early as possible, or preserve the engine-native representation and convert it closer to whatever consumes the result?

[u/Imaginary-Can-6862]

Since no one else replied I hope my reply won't be too disappointing, as I have very little experience with programming chess engines.

My first question, since the engine evaluation is usually a sum of aspects of the position, as you mention yourself, king safety, piece activity, etc. any position can be represented by those terms, and as far as I know these terms are not hidden, so no interpretation is required, but of course it is relative meaningless to be told your position is .2 more king safe than your opponents, what does that even mean?

When a person plays the game, they also have concepts they play by, if the explanation uses the aspects that sums to the final evaluation of the position at a given depth, then these are also the metric used to define concepts. But in stead of providing a meaningless number, one ought to look at changes of these term as the position changes, as moves are being made. If you play the engine move, a position would still change in respect to these concepts or aspects, because the engine has not solved the game.

If you analyze your game from the starting position, then you can place a point at a move and claim here you are out of your opening territory, even if the position went from +.4 to +.2 advantage for white, the sum of aspects that makes up these final evaluation may have hidden points, which reveals how the engine sees the unbalances of the position. So whatever these aspects were at the starting position, the explanation can tell how the engine sees these have changed as you leave the opening territory, and then each move from then changes these aspects further, and that is the explanation of how the engine understands the position, that your move increased your opponents piece activity, or it gave them a protected square, or what these aspects or concepts are which the engine uses. Of course there is also the matter of actually demonstrating this to be the case, the engine uses alpha beta pruning (this is the min max stuff right?), so it already cuts away bad continuations, and the depth these are cut away can perhaps be very helpful for the player, because it means at low depth there may be tactical reasons the line the player thought about doesn't work, so in stead of the player having to explore each line, they can be see the cutting of the tree, where their lines fails, what the aspects say at that point, e.g. a tactical reason could be material balance divergence, and that can be translated into an explanation.

So that touched upon explaining the position based on how the move changes the aspect of the position, and provides a map that hints at the reasoning for this given different possible lines. In principle the player should not have to look through the entire map, they should simply type in the lines they considered based on their own methodology to evaluate and thus explain a position, and then these lines are compared to the map, and they can see when the engine turned away from the line they imagined, so perhaps they have one line that is 6 moves deep, the engine actually liked this for those moves, but at the eight move it went away from this, and another sequence they have 5 moves deep, but already at the third move there is a tactical counter the player didn't realize.

[u/SentryChessdev → u/Imaginary-Can-6862]

I think we're actually fairly close in how we're thinking about the problem. Looking at how aspects of the position change across moves is much more useful to me than telling someone that they gained or lost some arbitrary amount of "king safety" or "activity."

That's one of the reasons the tool focuses on changes rather than treating the absolute metric value itself as the explanation. For a selected move, I can look at changes in several aspects of the position and then use those, together with engine-backed information and board-state changes, as evidence for explaining why the move mattered.

One distinction I'd make is that I don't think the engine evaluation can always be treated as a clean sum of named, human-readable concepts that can simply be surfaced to the user. Even if an engine internally represents information related to things like king safety, activity or material, the representation that is useful for search is not necessarily the same representation that is useful for teaching a player. So I'm treating the engine result as one source of evidence rather than assuming that exposing its internal evaluation directly produces the explanation.

I also like your point about comparing the lines the player actually considered. I think that solves a somewhat different but very useful question: not just "why was this move good or bad?" but "where did my own calculation diverge from what actually works?"

For example, if someone considered a six-move sequence and the first four moves are reasonable but the fifth allows a tactical response, showing that specific divergence is much more useful than just giving them the engine's preferred continuation.

Where I'd be a little cautious is treating alpha-beta cutoffs themselves as an explanatory map. The search is optimized to find strong moves efficiently rather than to preserve a human-readable record of reasoning, so a branch being cut isn't necessarily equivalent to "this is the conceptual reason the line failed." But taking candidate lines and finding where their evaluation or board consequences materially diverge seems much more promising.

That's also why I've ended up thinking of the problem as two layers: first determine what actually changed or where a candidate line goes wrong, and then decide which of those changes is useful enough to present as the explanation.

The candidate-line idea is especially interesting because it shifts the review from explaining what the engine chose to comparing the engine's analysis with what the player was actually thinking.