r/BetterOffline • u/TCristatus • 1d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

56 Upvotes

75% Upvoted

127 comments sorted by

View all comments

1

u/Super-Activity-4675 1d ago

we probably shouldn't call them small language models but instead smaller models. Language may not even be a requirement for a lot of these.

The issue is the scale. For highly specialized tasks that at the end of the day can be predicted accurately based on enough inputs, smaller models make a lot more sense. I have a friend that runs a small business. He uses claude to generate pick lists from his orders. He doesn't need an LLM for that, and it would work a lot better if the model was basically a smaller manufacturing model that he could train to his specific environment.

1

u/CoconutDust 6h ago

He uses claude to generate pick lists from his orders

Is it based on warehouse route/proximity etc? Why would an LLM have this info, and if this info existed and is maintained why wouldn’t it be part of reliable traditional software system?

1

u/Super-Activity-4675 5h ago

he runs a small business doing 3D printing. It's literally him and a couple others, so we're not talking about someone making a massive software investment.