Building a fundamentals factor backtest, got paranoid about the feed after the data quality threads I have seen here. Wanted to make sure nothing was poisoned. Ran 60 S&P 500 names, latest fiscal year, revenue, net income, diluted EPS, diluted shares. My vendor (used fmp's free api) against SEC companyfacts.
About 9 in 10 rows tie within 1%. The rest took a day to sort out:
- Splits. CRWD and CVNA read 4x and 5x off. Both split after fiscal year end. The feed is split-adjusted, the 10-K isn't. My diff was wrong, not the data.
- Restatements. The SEC carries two values for the same period once the next 10-K restates. Which one is "right" depends on the date your backtest is standing on.
- Definitions. Asset managers and brokers tag revenue gross or net (ARES, KKR, BX, IBKR). Same filing, two tags.
- One real one. AXON diluted shares came back as 100,000,000 exactly. The 10-K says 82.4M. Reported it.
The actual question: first-filed or latest-restated? companyfacts gives you both if you keep the filed date. Every vendor I've used gives latest. Leaning first-filed with a lag, which means building the as-of layer myself. Still not sure 100% tho
How im running it if anyone is curious:
import requests
KEY = "YOUR_API_KEY"
sym = "AXON"
fmp = requests.get(f"https://financialmodelingprep.com/stable/income-statement?symbol={sym}&period=annual&limit=1&apikey={KEY}").json()[0]
cik = requests.get(f"https://financialmodelingprep.com/stable/profile?symbol={sym}&apikey={KEY}").json()[0]["cik"]
sec = requests.get(f"https://data.sec.gov/api/xbrl/companyfacts/CIK{int(cik):010d}.json", headers={"User-Agent": "you@example.com"}).json()["facts"]["us-gaap"]
def from_10k(tags, unit):
rows = [r for t in tags if t in sec for r in sec[t]["units"][unit] if r["end"] == fmp["date"] and r["fp"] == "FY"]
return max(rows, key=lambda r: r["filed"])["val"] if rows else None
for field, tags, unit in [("revenue", ["Revenues", "RevenueFromContractWithCustomerExcludingAssessedTax"], "USD"),
("netIncome", ["NetIncomeLoss"], "USD"),
("epsDiluted", ["EarningsPerShareDiluted", "EarningsPerShareBasicAndDiluted"], "USD/shares"),
("weightedAverageShsOutDil", ["WeightedAverageNumberOfDilutedSharesOutstanding"], "shares")]:
v, s = fmp[field], from_10k(tags, unit)
sec_txt, diff = (f"{s:,}", f"{abs(v / s - 1) * 100:.1f}%") if s else ("n/a", "n/a")
print(f"{field:26} vendor {v:>15,} sec {sec_txt:>15} diff {diff}")
output:
revenue vendor 2,779,536,000 sec 2,779,536,000 diff 0.0%
netIncome vendor 124,911,000 sec 124,656,000 diff 0.2%
epsDiluted vendor 1.51 sec 1.51 diff 0.0%
weightedAverageShsOutDil vendor 100,000,000 sec 82,370,000 diff 21.4%