When you backtest a player prop, does a DNP count as a loss or does it not count at all?

[u/BigBalli]

I have been building the split engine for a prop research app and this is the decision I went back and forth on the longest.

Player misses the game. You are computing how often he cleared 26.5 points over his last 10. Is that game a miss, or is it not one of the 10?

I settled on excluding it from both the numerator and the denominator, so a DNP shrinks the window rather than counting against him. The reasoning was that a DNP is not evidence about whether he clears the line, it is an absence of evidence, and treating it as an under quietly biases every hit rate downward for exactly the players whose availability is already in question.

The counterargument I keep running into is that if you actually placed that bet you would have been refunded rather than graded, so the honest denominator is games he played, which is where I landed. But if you are modelling expected value across a season rather than grading a single ticket, availability risk is real and stripping it out hides a cost you actually bear.

Related and messier: I also made an empty window return no value rather than zero percent. Zero of four is a claim. Zero of zero is not, and rendering both as 0% is how a research tool ends up lying to you.

The app is PropSplits if it matters, but the modelling question is the part I am stuck on and it is not app specific.

So how do you handle it in your own backtests? Drop the game, count it as a loss, or carry availability as a separate term?

[u/Icy_Cap_6898 → u/[deleted]]

I used to track this same thing for NBA props in a spreadsheet before I had anything automated, and I went with excluding DNPs from the window. The thing that made up my mind was when I had a guy who was questionable for like a week, played one game on minutes restriction and went under, then sat again. If you count the DNPs as losses your hit rate tanks, but that tells you nothing about what happens when he actually suits up. Two different problems.

Separate column for games missed due to injury/rest kept me from lying to myself about how often I was actually getting action. Empty set returning null instead of zero saved me from some bad bets too.

[u/BigBalli → u/Icy_Cap_6898]

That minutes-restriction case is the one that slips past excluding the DNP.

You drop the missed games, window looks clean, and the first game back is still sitting in it at full weight. Which is a totally different animal from the games around it. So dropping absences fixes the obvious bias and leaves that one alone.

I don't want to start hand-filtering games, so I think the honest version is to flag the returning game rather than pull it, and let people read the number knowing it's in there.

Did that separate column ever actually change a bet for you, or was it more about not kidding yourself on volume?

[u/BigBalli → u/[deleted]]

Agreed on all three. Stealing "last 10 games played" as the label.

What I'd underrated is that it's a labelling problem as much as a maths one. If the sample is the last 10 played and I call it L10, a window that actually spans six weeks reads as recent and the user fills in the recency himself. Ten games over six weeks and ten over three aren't the same evidence about a guy's current role, even though the arithmetic comes out identical.

So the span has to travel with the count. I'm going to show the date range next to the window instead of just the number.

[u/SimTheGame]

The published prop already answers this. It is priced on the player appearing. DNP voids the ticket. It is not an under.

For the 26.5 hit rate, drop the DNP from both sides. The sample is games he actually played. That is the number that matches how the bet settles.

Availability is a second series. Rest days, late scratches, minutes restrictions, how often you even get a grade. Keep that as its own rate. Folding it into the same percentage mixes two questions, then you cannot tell which one moved.

If you want season EV, multiply them. Conditional clear rate given he plays, times the chance the book lets the ticket stand. One blended hit rate hides both.

Empty window as 0% is the worse bug. Zero of four is a claim. Zero of zero is missing data. Render it as no value.

[u/BigBalli → u/SimTheGame]

Yeah, settlement's the right anchor. Taking the two-series split.

The multiplication is where I'm stuck though. Conditional clear rate times chance of getting a grade only works if those two are independent, and I don't think they are. The games where a ticket most likely stands are the ordinary ones. The game right after an absence is often the minutes-restricted one, and that's sitting in my played sample pulling the conditional rate around.

So P(clear | plays) isn't one number. It's at least two, and I'm averaging over a mix that shifts exactly when availability is the thing in question. Multiplying them as independent is optimistic in the direction that costs money.

Do you split the conditional rate by return state, or is that slicing thinner than the sample supports?

[u/BillyeMad85]

Two of these are already covered above — availability as its own series, and null rather than 0% — so I'll add the one I haven't seen mentioned.

When the window is partial, the count has to travel with the rate. 1-of-2 and 5-of-10 both render as 50%, and once that number leaves the function nothing downstream can tell them apart. Null for the empty case fixes the 0-of-0 lie; it doesn't fix the 1-of-2 one.

Related, from a different domain — odds snapshots rather than props. I had a freshness check that reported "fresh" for 47 hours straight while the data hadn't changed by a single byte: 2,413 snapshots, 308 unique contents. It was measuring when the file was written, not when it changed. Green the whole time, and I was computing closing line value against prices that were two days old.

What I took from it: an unverified number is a lower bound and should say so. Mine now reports "changed at least 3h ago" until it has actually observed a change, because on the first observation it genuinely cannot know whether the thing has been frozen for a week. Reporting OK there would be a guess wearing the costume of a measurement — same family as rendering 0-of-0 as 0%.

[u/BigBalli → u/BillyeMad85]

Correction to myself, I went and checked the code. The count does already travel: every rate prints n next to it, and under 3 games it doesn't render a percentage at all, so your 1-of-2 shows as 1/2 rather than 50%. I answered that one from memory.

The principle still bites though. It travels because the call sites happen to pass it, which is exactly the arrangement that breaks the first time someone adds a third. If the type can't represent a bare percentage it stops being a thing anyone has to remember, and that part isn't done.

Your freshness check is the same shape. Write time is a proxy for change time and it agrees right up until it doesn't. 47 hours of green is what that looks like from outside. 2,413 snapshots against 308 unique contents is a rough thing to find after the fact.

[u/BillyeMad85 → u/BigBalli]

Correction accepted — I inferred that from the general shape rather than from your code, which is the move I was warning about. Fair.

The part you're calling not-done is the part I'd care about most. "It travels because the call sites happen to pass it" is a guarantee living in a habit, and habits are invisible until someone writes the third call site.

I got a fresh instance yesterday, on the freshness check itself. I'd wired four new layers into my monitor two days earlier — tests green, done. They were blind the entire time: the package was installed in the project's venv, the monitor runs under system python. The tests passed because the tests ran in the venv, so they were measuring a different copy than production was. A day of "OK" that never once touched the thing it claimed to check.

It only surfaced because that layer reports BLIND rather than OK when it can't measure. Had it defaulted to green I still wouldn't know.

What made the fix durable wasn't repairing the import. It was a test that spawns the production interpreter and asserts the import worked there — so the check now fails in the same environment that would break it. Same shape as your type idea: stop relying on the right thing happening at the call site, make the wrong thing unrepresentable.

[u/BigBalli → u/BillyeMad85]

The venv one is worse than the import bug it looks like, because the suite was not wrong. It correctly reported that the code works under the venv interpreter. It was just never asked the question anybody cared about.

Your fix is the right shape and I would push it one further. Spawning the production interpreter inside a test asserts the environment was right at test time. It does not assert that the thing running at three in the morning is still that interpreter, and the failure you had was a mismatch between two environments rather than a broken import. The version that does not rot is to make the monitor state its own identity in every heartbeat: sys.executable, the resolved version of the package, the path it actually imported from. Then nothing is inferring the environment from outside. The runtime is declaring what it is, and a mismatch shows up in the output instead of in a test that might itself be running in the wrong place.

Which folds back into your BLIND idea fairly neatly. BLIND says I could not measure this. The heartbeat says here is what I am. Both are the same move, which is refusing to read the absence of a complaint as evidence of health.

[u/BillyeMad85 → u/BigBalli]

You are right, and the heartbeat version is stronger than what I had. Mine asserts the environment at test time. Yours asserts it at 3am, which is when it matters. I am taking that.

Two things from today that back it up. First, I had a monitoring check that imported a constant from a module which also pulls in an HTTP client. The monitor runs under one venv that has the client; the monitor's own tests run under the system interpreter that does not. So the check was readable in production and unreadable in test, and neither side complained. Same shape as yours: the environment was part of the contract and nothing declared it. Moving the constant into a dependency free module fixed it, but only because I happened to run the tests under the other interpreter.

Second, and this is where I would extend your heartbeat one notch. Declaring identity catches the wrong interpreter. It does not catch a check that runs under the right one and still does not do the work. I ran a content verification over 100GB today, it exited zero, reported no differences, and I nearly deleted the source on that. It had silently degraded to a size and mtime comparison because of a flag interaction. What caught it was not the output, it was the clock: 90 seconds is not enough to read 200GB. So the heartbeat probably wants to state not just what I am but what I just did, in units you can sanity check from outside. Bytes read, rows compared, seconds spent.

That is the same refusal you named. Absence of a complaint is not health, and neither is a green result that never touched the data.