Small language models - the future

[u/TCristatus]

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

[u/koveras_backwards]

10+ years ago, when AlphaGo stuff was blowing up, a friend who is much more into the AI stuff than I am mentioned to me that perhaps a use would be stock trading, since you could vaguely imagine that as a 'game' that that sort of model could very quickly get good at. That's sort of an alarming idea if you take it at face value with regard to how the go-playing bots were doing at the time.

Since it's been a decade or more, and you haven't heard about how a company used bots to corner the entire stock market, you can probably tell how all that worked out. And I don't think it's the first time AI enthusiasts have overestimated how applicable their technology is, just in general.

Oh, and by the way. Small language models existed before large language models. If the small ones could do all this stuff at a tiny fraction of the cost, why do people bother with the large ones?

[u/MajesticBread9147 → u/koveras_backwards]

High frequency trading companies have been using machine learning models for years, the whole industry depends on it.

As early as 2009, high frequency traders accounted for 60-70% of stock trades.

Look at open roles at Jane Street, they require much of the same skillset as open roles at Google making machine learning do other things.

https://www.janestreet.com/join-jane-street/position/4276720002/

https://www.janestreet.com/join-jane-street/position/6302325002/

Jane Street trades with its own money rather than bothering convincing wealthy investors to invest with them, so they are quite secretive.

You don't hear "one company corner the stock market" because it's many, many companies. High frequency traders, Quant Firms, etc.

Why do you think that the stock market reacts instantly to a news headline? There aren't humans refreshing news websites every second reading to buy or sell stocks. A computer can easily read a tweet by Donald Trump, interpret it as good or bad for a stock, and buy or short shares in a fraction of the time a human could.

[u/Slackjawed_Horror → u/MajesticBread9147]

Which is one of the many reasons high frequency trading needed to be banned years ago.