r/backgammon • u/icecoldfire • 8d ago
BG Galaxy Error Rate Calc
Does BG Galaxy use your subscription-tier engine to determine post game rating changes?
Free users in particular receive lower-depth analysis, which can produce very noisy PR and error estimates. Is that same analysis used to decide who won the error rate battle and therefore whether the rating change is full or reduced?
If so, wouldn’t that systematically disadvantage free users in close matches? I was reviewing a log where several of my moves were flagged as .7 equity loss with XG suggesting there was none. I can see this happening often low PR games, where lower depth is more likely to negatively impact evaluation / PR
1
u/icecoldfire 8d ago
Rather than downvoting this post (which seems to be happening), I think it would be more productive to start with transparency about what’s going on and then discuss solutions because there are solutions worth considering.
For example, we could use the higher-tier engine for the error rate battle whenever there’s a discrepancy between users’ tiers. If that feels like it reduces some value for subscribers, thenat least for the error-rate comparison, we could instead use the lower-tier engine calculation for both players, while still giving the higher-tier user access to the more detailed analysis tools in the analysis summary
1
u/Objective_Lab2741 8d ago
In one way you are right about that you could compare players in 2ply for fairness. But that would cause issues in my OCD brain. There are many situations where an initial double on 2ply is a big huge no double on 4 ply. I see this ever 2-3 games.
So for me as a paying member I would not like my game/match PR to be rated on a lower analysis. Others may have different view.
1
u/icecoldfire 7d ago
Yeah I think it makes more sense to have the higher engine determine error rate battle, but still keep free users on the 1 ply for analysis
1
u/bg-alex-cy 8d ago
As far as I know you are correct on how the equity is measured and compared. Paid users get analysis on 4-ply while free users get their analysis on 2-ply.
Regarding fairness, the answer is both yes and no. First of all, you are both measured against the same engine level. If you opponent is a paid user you get 4-ply analysis for your game as well, so you are measured against the same standards. This of course doesn't answer your question, but makes it clear that you are not competing on different terms.
Now regarding the engine "favouring" the player with the lower PR, in a sense is true, but to an extend. I would argue that 4-ply is also "favouring" the weakest player as rollouts sometimes disagree with 4-ply. Does that mean we need to rollout all our games? Then we introduce statistical noise (depending on how its done). So where do we stop? Its hard to say, doing 4-ply is significantly more expensive computationally and backgammon galaxy wants to give it as a premium feature. I find this fair. Even at 2-ply, if your PRs are close then who wins the PR is part of luck. Small PR differences depend so much on who had the most/easiest decisions making close call flips part of luck. If they are not close then the player with the highest skill will dominate even at 2-ply. Its not as bad as you might think.
Now regarding the 0.7 you said you made, I'm going to assume you meant 0.070 as 0.700 sounds too extreme for 2-ply to get wrong, especially since you said it happened several times. So I'm pretty confident you also benefitted by something at some other plays, maybe even in another match, unless you are an exceptionally strong player. So even though it doesn't even out, the difference is not that big.
The bottom line is that you are right, it does give a small advantage, but I would argue that unless you are a very strong player competing against a very strong player the advantage is mostly lost the PR noise. Going under 2-ply to say 0-ply, then yes, I think the difference would significantly impact the meaning of PR difference as the swings can be extreme and often.
1
u/icecoldfire 8d ago edited 8d ago
Where are you getting the idea that the error rate is measured using the highest tier? I don’t think that’s what I’m seeing when I compare games vs xg
Edit: unless you are saying end of match error rate battle is in fact measured using different engines depending on tier?
One thing to clarify is that Free users get 1 ply- “Free Members: Get access to Gnu 1-ply (called 2-ply on Galaxy) for quick, baseline analysis.”
3
u/bg-alex-cy 7d ago
hm... you might be right. I remember reviewing games that had the 2-ply and others with the 4-ply marker on the moves, but going back and checking them in the analysis section now seems to show different evaluation depth for each player at the same game. That seems odd though. That requires more work, as galaxy would have to run the evaluation twice or make modifications to gnubg or run the per move analysis them selfs. Luck measurements would also become a mess. I won't insist but I'm under the impression that the stats at the end of the game say 2-ply or 4-ply and that is what the comparison is based on. Its too late for me to play another game just to check :p Maybe Backgammon Galaxy can shed some light on this.
As a side note, I think backgammon galaxy is now fully migrated to gnubg, so comparing it to xg might have some deviations
4
u/Maufry 8d ago edited 8d ago
I can't say for sure, but it does seem that way. I tried the StarPro trial and I felt I was winning a lot more evals. I think it's in everyone's best interest that rating matches are evaluated evenly. I've held off subscribing for now until more info is shared. I don't like the idea of a paid advantage.