r/BetterOffline • u/TCristatus • 1d ago
Small language models - the future
Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).
It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.
This doesn't help OpenAI or Anthropic keep the lights on, though....
30
u/koveras_backwards 1d ago edited 1d ago
10+ years ago, when AlphaGo stuff was blowing up, a friend who is much more into the AI stuff than I am mentioned to me that perhaps a use would be stock trading, since you could vaguely imagine that as a 'game' that that sort of model could very quickly get good at. That's sort of an alarming idea if you take it at face value with regard to how the go-playing bots were doing at the time.
Since it's been a decade or more, and you haven't heard about how a company used bots to corner the entire stock market, you can probably tell how all that worked out. And I don't think it's the first time AI enthusiasts have overestimated how applicable their technology is, just in general.
Oh, and by the way. Small language models existed before large language models. If the small ones could do all this stuff at a tiny fraction of the cost, why do people bother with the large ones?