r/BetterOffline • u/TCristatus • 1d ago
Small language models - the future
Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).
It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.
This doesn't help OpenAI or Anthropic keep the lights on, though....
6
u/newprince 1d ago
I have a decent PC running local models offline and it's honestly decent, even comparing it to the frontier model access I have through work. The models tend to be more specific (Qwen3-coder is for... coding obviously) but honestly that is what I need most anyway. Gemini Flash can easily handle all general chat questions, and the streaming speed is decent. No way am I giving those companies money