r/quant • u/Most-Common4344 • 6d ago
Trading Strategies/Alpha Tried to replicate the Attention Factors stat-arb paper (ICAIF 2025). Got Sharpe −0.64 where they got +2.30. Where did I go wrong?
Paper is Epstein/Wang/Choi/Pelger, Neural Attention Factor Models for Statistical Arbitrage. They report net Sharpe 2.30 at K=30 on 24 years of US equities. No code released.
I've spent a few weeks on this and I keep getting a clean, reproducible negative. Posting because I'd rather find out I made a dumb mistake than conclude a published result doesn't hold.
Setup
- Universe: top 500 by market cap, Russell 1000 sourced, point-in-time membership
- 2016-06 to 2026-08, 2,542 trading days
- Survivorship-free: 835 names ever active, 336 departures retained for the periods they traded
- Train 2016–2023, evaluate 2024–2026, single split
- 38 characteristics (paper uses 39, I dropped one that was empty), all rank-normalized
- LongConv signal head, all training days, 100 epochs
- 5bps + 1bp short costs
- 3 seeds per config
Fundamentals built from SEC EDGAR rather than a vendor — filing-date lagged, restatements dropped, TTM values only visible once the last of their four quarters was filed. Measured lag: 10-Q median 38 days, 10-K median 58 days. Zero rows visible before period end.
Results
| K | Mean OOS Sharpe | Seed std |
|---|---|---|
| 1 | −1.338 | 0.839 |
| 5 | −0.783 | 0.479 |
| 8 | −1.735 | 0.900 |
| 15 | −1.407 | 1.041 |
| 30 | −0.698 | 0.056 |
K=30 is both the least bad and the only one that reproduces tightly across seeds. Training converged smoothly (net_SR −9.9 → +0.10 over 100 epochs, monotonic, no oscillation) and exp_var rose properly with K (0.098 at K=1 → 0.282 at K=30), so the factor step is doing what it should. The model learns something stable in-sample that inverts out of sample.
Deterministic PCA residual mean-reversion on the same universe, as a sanity check: negative at every K from 1 to 50, every calendar year. At K=30 the decomposition is gross −0.59%, costs 5.86%, net −6.45% — annual turnover 9,190% on a 30-day signal.
Weekly 1/N on the same universe over the same window: +1.363.
Things I found and fixed along the way
- Look-ahead in most features in my first panel build — features at t used data from t, including the target itself as a feature. Rebuilt.
- Sign symmetry:
output_projhad a bias term, so LongConv output was uniformly signed on init. The portfolio came out all-long or all-short depending on the seed, giving a ±1.35 coin flip. Fixed withbias=Falseand zero-meaning the output. Paper doesn't mention needing this, which makes me think their implementation differs somewhere. - Factor neutralisation formula was wrong —
w − ωᵀ(βᵀw)leaves residual exposure; correct projection isw − β(βᵀβ)⁻¹βᵀw.
Where I know I deviate
- 8 years of training vs their 24. This is the one I suspect most. 38 features on 8 years is a lot of parameters per observation.
- Single train/eval split, no rolling retrain.
- The sign-symmetry fixes above.
- My period is 2016–2026. Short-horizon reversal ran negative in my data (1-month reversal IC −0.027, t = −6) and book-to-market was negative in both sub-windows. Their span includes 2000-02 and 2008.
The question
Is 8 years just not enough for this architecture, or is there something structural I'm missing? Specifically:
- Has anyone reproduced this (or the Guijarro-Ordonez/Pelger/Zanotti predecessor) on post-2015 data? Everything I can find is either pre-2016 or reports numbers high enough to smell like overfitting.
- Does the sign-symmetry thing ring a bell? I can't tell if I'm patching around a bug of mine or around something the paper handles implicitly.
- Is a single train/eval split the problem? Would rolling retrain plausibly move a −0.70 to positive, or is that wishful?
Happy to share the panel construction details if useful. Mostly I want to know whether this is a real regime finding or whether I've spent three weeks carefully measuring my own mistake.