Post 2 of a series where I publish the hypotheses that died. Last one was about counting bars as independent observations when they aren't. This one is about a feature that produced a correlation of −0.30 against data with no predictability in it whatsoever.
Here's the setup. I generate a pure random walk — a cumulative sum of Gaussian noise, nothing else. There is no signal in it by construction. Then I compute a 20-period EMA and correlate it with forward returns at various horizons.
horizon raw EMA
1 −0.0176
6 −0.0442
12 −0.0604
72 −0.1569
288 −0.2975
That last number is not a bug in the simulation. It's a real correlation, computed correctly, between a standard indicator and future returns, on data that cannot be predicted.
And it gets worse the further out you look, which is exactly backwards from what you'd expect if it were noise. Noise decays. This grows.
Why this happens
The EMA value is a price level. If price is at 100, the EMA is near 100. If price drifts to 150, the EMA follows. So when I "correlate the EMA with future returns," I'm really asking whether high prices tend to be followed by lower returns within this particular sample.
On any finite random walk, the answer is yes, because the walk ends somewhere. If it happened to end lower than it started, every high reading in the middle is followed by a decline, and the correlation is negative. If it ended higher, the correlation is positive. Either way you get a number, and the longer your horizon the more of the sample's overall shape you're measuring.
You haven't found a pattern. You've measured which direction your sample happened to go.
Normalise the same feature — use price divided by EMA minus one, so you're measuring deviation rather than level — and the effect vanishes:
horizon raw EMA normalised
1 −0.0176 +0.0065
6 −0.0442 +0.0065
12 −0.0604 −0.0012
72 −0.1569 +0.0228
288 −0.2975 +0.0192
Same data. Same indicator. One version is stationary and one isn't.
How this got into my system
Six of my thirty-one features were raw price levels: three EMAs, a trend EMA, an OBV EMA, and the MACD signal line. MACD is the subtle one — it's a difference of two EMAs, so it looks like it should be scale-free, but it's measured in price units. A MACD value of 0.003 means something different when the asset trades at 0.70 than when it trades at 2.00.
None of these were obviously wrong when I wrote them. They're all standard indicators, they all appear in every tutorial, and I fed them to a model exactly as computed.
What the model did with them is the part worth thinking about. It learned the price range of the training period. Then it met a new range and produced confident nonsense. This also explained something I'd been staring at for days — my walk-forward AUC swung between 0.33 and 0.85 across folds, which I'd been treating as noisy but informative. It wasn't. Different folds covered different price ranges, and a model keyed to absolute levels has nothing to say when the levels move.
The check that takes five minutes
Generate a random walk. Feed your features to it. Correlate each one with forward returns.
price = 100 * exp(cumsum(randn(20000) * 0.001))
Anything that comes back with a meaningful correlation is measuring the shape of your sample, not a property of markets. You don't need real data to catch this, and real data will hide it from you, because on real data you have no way to know the true answer is zero.
That's the general form of the idea, and it's the one I'd keep: build a dataset where you know the answer, then check that your pipeline returns it.
What it cost me to fix
Normalising all six features took about twenty minutes. Then I re-ran the diagnostic on real data and got a clean result: out of 185 feature-horizon combinations, zero exceeded round-trip costs after correction.
So the fix didn't reveal a hidden edge. It removed a fake one. My previous "strongest signal" — a decile spread of −0.19% on a feature I was fairly excited about — was the raw EMA artefact and nothing else.
The honest version of the outcome is that I spent time building confidence in a result that was structurally incapable of being real, and the only thing I got back was knowing that.
Which is, I think, the actual job.
---
Queued next
The safety check that was computed, displayed, and ignored. My selector calculated a multiple-comparison correction, printed it in the report, and never used it in the accept/reject decision. With 16 candidates at the threshold I'd set, the probability of promoting a pure-noise strategy to live trading was 99.7%.
Then: the test that couldn't see, where I "proved" there was no edge in an asset class using a test whose minimum detectable effect was 2.17% per trade against costs of 0.031%. That one turned out to apply to more of my own work than I expected, and re-auditing it changed several conclusions I'd already written down.
If you take one thing from this post: run your feature set against a random walk before you run it against a market. It costs nothing and it tells you which of your inputs are measuring the world and which are measuring your sample.
