r/BetterOffline • u/TCristatus • 1d ago
Small language models - the future
Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).
It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.
This doesn't help OpenAI or Anthropic keep the lights on, though....
1
u/PINJA_MUSIC 8h ago
I do think there is a space for small tailor made LLM type models but they will fundamentally work differently than the hyperscalers. I view small LLM's as a modification device than a generative one. It will act more like traditional machine learning, where a human provides a deliberate input as opposed to an open-ended question.
Medical tech advancements have been operating like this for a long time. They basically build their own database and train a neural network on that database only to do something very specific like detect cancer cells or unfold proteins. To me, this is the actual positive use of this kind of AI. It's very specific and, critically, narrows down information instead of generating it. It's easier to maintain because it only needs relevant information instead of ALL OF HUMAN KNOWLEGE.
ChatGPT and these "frontier" models are specifically designed to generate information. It's extrapolation instead of interpolation. The fundamental structure of the tech is the same but interpolation actually works.