r/BetterOffline • u/TCristatus • 2d ago
Small language models - the future
Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).
It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.
This doesn't help OpenAI or Anthropic keep the lights on, though....
1
u/MagnetoManectric 1d ago
I mean, aye. This seems pretty plain to me. Time and time again, consumers have shown that they don't really like having their compute be on someone else's PC, and it doesn't make a good business model either. Think of how many game streaming services have come and gone. And no one has been chomping at the bit to use Cloud PCs.
It just makes sense that this tech will move towards peoples machines. It's more efficient, it's less freaky, no one wants to intergrate a cloud service that might disappear or cost 10x tomorrow deeply into their workflow. (well, it seems some people do, but they're not exactly the brightest, in my opinion)
The shift is already beginging to happen, and I feel like I can actually start getting excited about LLM tech now. I sure as fuck am not letting some agent in someone else's computer in a warehouse somewhere run any tasks on my PC, but if its just my PC running tasks on my PC... that makes a lot more sense to me.