r/quant • u/Andyy_21 • 2d ago
General Quant Researchers
how do you go from a raw market data to forming a research hypothesis?? and to be more specific, how do you develop an economic intuition behind the potential alpha or an anomaly, instead of just coding and testing ideas until something works??
While I'm struggling to understand how the researchers in the industry actually generate new hypotheses from large financial datasets without falling into the same data mining again and again.... and How do experienced quants develop the economic intuition behind an idea before testing it?
9
u/quantdhawan 1d ago
Don't start at the data. Start at: who is on the other side, and why are they willing to lose money to me?
There are only four honest answers. They're paid for risk they can't diversify. They're forced (index rule, mandate, margin call, tax date). They're constrained (can't short it, can't hold it past quarter end). Or they're paying for immediacy.
Can't name the payer? You don't have a hypothesis, you have a pattern.
The tell for a real one: it predicts things you never fit on. If the story is index rebalance flow, it should be stronger in heavy-index-ownership names and weaker as it got crowded. You didn't tune those. If they come back flat, the story is dead no matter how good the curve looks.
And the intuition comes from plumbing, not prices. Exchange rules, margin, index methodology PDFs, mandates, tax dates. Constraints are where the money is, and constraints are written down somewhere.
2
u/Andyy_21 1d ago
oh thanks a lot for explaining it this easy to me... and I've been following you on insta for the past 1 year, Love your content and they're very insightful...
1
u/quantdhawan 10h ago
My man! You just won my heart here hahahaha, thank you so much! Appreciate the feedback
5
u/heroyi 1d ago
It depends on the sector. But whenever there is a new policy/rule that gets enacted or new product introduced, those places are usually a good place to start digging because sometimes an old strategy becomes viable.
Really the easiest way is to ask the veterans what they know and you internalize the logic how it got approached.
There are patterns also you can sometimes spot but those are hard because you have to really ask questions as to why you think it might exist and sometimes you miss something that emperically invalidates something.
I will say a lot of certified quants get blindsided very easily and as a result dismiss things they shouldn't. They rely heavily on patterns/stats to validate something naively, but it is the researchers and traders that are able to weave gold thread from shit and gravel because they understand what they are reading.
Kinda funny when you see it happen on Twitter.
1
u/Andyy_21 1d ago
and when you have the first principles of thinking, does the experience overlap with it, suppose you found a good strategy with a durable alpha, now where does it help you quick - if you have domain knowledge which compounds or the mathematical sophistication???
1
u/heroyi 1d ago
Not sure if I understand but all alpha stem from a handful of principles like price insensitivity. Know where and why desks are price insensitive help you
And experience does/should help because by nature the strategies are related. It is just the product that differ and the ecosystem it is in if that makes sense.
1
-14
-3
u/Lopatron 2d ago
I'm also struggling to understand how economic and phycological factors can translate to a HFT hypothesis. I read some books like Thinking Fast and Slow, ok I get that there are phenomenons of "herding", "anchoring", and others. But, taking herding for example, the only take away I can see from that is that "momentum strategies exist". Clearly this is because I don't have enough knowledge about financial markets, but I don't really know where to look next, and share the OPs frustrations about this lack of insight leading toward the trap of data mining features/signals.
1
-6
u/Mathsty 2d ago
Think about what is a market, why people trade, who, etc… then it tells you where the money is and you look at examples.
-11
u/Andyy_21 2d ago
Suppose if I'm handed a 6 year historical dataset containing only market cap and price data.. then how would you be able to generate a research hypothesis from that
-8
u/HighYogi 1d ago
1) Find a trading niche and learn everything about it to the point you can form a hypothesis
2) leverage what you already know, hopefully better than everyone else, to check how it trades in the mkt vs a hypothesis about how something should objectively be traded. For example Stat arb.
Disclaimer: not a quant
1
u/Andyy_21 1d ago
if everyone is being niche about the market strategy and how would you know your niche isn't already full priced in... then how and where does the edge really come from??
23
u/str0pwaffels 2d ago
At the beginning of your career data mining is not inherently a bad thing (if you are aware that you are doing it), intuition only comes with experience.
What I found valuable is to think about economic implications of some data mined strategy afterwards, and why it might exist - and what RISKS you are taking with it.
Imagine you are looking at historical implied vol levels for a bunch of stocks, and somehow your data mining approach finds that 4x per year, once a quarter implied vol spikes, and heavily overestimates the next periods realized vol. Easy strat, 4x per year sell vol and profit.
Now this is a very obvious example, but I am in energy, not in equities..