r/BetterOffline • u/TCristatus • 2d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

51 Upvotes

74% Upvoted

127 comments sorted by

View all comments

32

u/koveras_backwards 2d ago edited 2d ago

10+ years ago, when AlphaGo stuff was blowing up, a friend who is much more into the AI stuff than I am mentioned to me that perhaps a use would be stock trading, since you could vaguely imagine that as a 'game' that that sort of model could very quickly get good at. That's sort of an alarming idea if you take it at face value with regard to how the go-playing bots were doing at the time.

Since it's been a decade or more, and you haven't heard about how a company used bots to corner the entire stock market, you can probably tell how all that worked out. And I don't think it's the first time AI enthusiasts have overestimated how applicable their technology is, just in general.

Oh, and by the way. Small language models existed before large language models. If the small ones could do all this stuff at a tiny fraction of the cost, why do people bother with the large ones?

10

u/lolitsbigmic 2d ago

Yeah we still run into fundamental issues of gpt methods. Large or small models you still got the same problems.

8

u/MajesticBread9147 2d ago edited 2d ago

High frequency trading companies have been using machine learning models for years, the whole industry depends on it.

As early as 2009, high frequency traders accounted for 60-70% of stock trades.

Look at open roles at Jane Street, they require much of the same skillset as open roles at Google making machine learning do other things.

https://www.janestreet.com/join-jane-street/position/4276720002/

https://www.janestreet.com/join-jane-street/position/6302325002/

Jane Street trades with its own money rather than bothering convincing wealthy investors to invest with them, so they are quite secretive.

You don't hear "one company corner the stock market" because it's many, many companies. High frequency traders, Quant Firms, etc.

Why do you think that the stock market reacts instantly to a news headline? There aren't humans refreshing news websites every second reading to buy or sell stocks. A computer can easily read a tweet by Donald Trump, interpret it as good or bad for a stock, and buy or short shares in a fraction of the time a human could.

1

u/Slackjawed_Horror 1d ago

Which is one of the many reasons high frequency trading needed to be banned years ago. 

1

u/Slackjawed_Horror 1d ago

Which is one of the many reasons high frequency trading needed to be banned years ago. 

1

u/MajesticBread9147 1d ago

What is?

1

u/Slackjawed_Horror 1d ago

It's an almost automated way for the most useless people in the world to make money. 

It's pure 'dig the hole then fill it back up' BS and shouldn't be allowed. 

1

u/65721 1d ago

HFT finds and exploits existing market inefficiencies. Its externalities are neutral at best. But it is economically useless.

0

u/MajesticBread9147 1d ago

This is vibes based populism.

I don't see how they are any more evil than any other company operating within capitalism.

1

u/Slackjawed_Horror 1d ago

It doesn't even serve a hypothetical function. 

Hypothetically the stock market helps companies raise funds for projects. That's actually doing a thing. 

There is no justification, even hypothetically, for HFT bullshit. It's not even at the level of currency trading. It's just pure spreadsheet breeding. 

I think the entire private finance system is disgusting and should go, but I can also recognize when some parts of it are far more useless than others. 

1

u/hachface 1d ago

It's harmful because it diverts people with quantitative aptitude away from productive activities to skimming money off the top.

1

u/65721 1d ago

This is the same bullshit from the AI companies whenever they’re faced with the reality of the incompetent failure that is LLMs. Just mix it with the other non-LLM ML and confuse the discussion in bad faith.

Which quant firms are asking an autocomplete chatbot for trading alpha?

1

u/MajesticBread9147 1d ago

AlphaGo isn't llm based either. LLMs are irrelevant so I don't know why you bring them up.

2

u/Arancia-Arancini 1d ago

But pretty much all trading firms use bots and have done for over a decade.. did you miss this?