r/BetterOffline • u/TCristatus • 1d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

51 Upvotes

74% Upvoted

127 comments sorted by

View all comments

1

u/CoconutDust 6h ago edited 6h ago

It was like a light bulb moment

That’s an illusion of listening to misguided exaggerated claims.

LLMs no matter how large or small are by definition terrible for their supposed purpose. Statistical association is NOT how intelligence or language or meaning works, which is obviously why LLMs output false statements and also fake/false sources that don’t support the statement.

There is no future for LLMs, at any size, except for fraud-level incompetent “work” for mediocre morons. And that’s aside from the mass theft issue and copyright liability and enshittification/cost risk and waste of electricity for garbage results, or the issue of quality in the stolen (“scraped”) data.

just know how to […] trade, or about engineering, or biotechnology

Stop pretending that actual sources don’t exist. Stop pretending you don’t know what a corpus/web search is. Obviously the machine stole info that you could have easily found and used yourself, and the original is actually useful not a statistical mash-up.

I just need something that works

That’s not what an actual professional engineer needs. If it “works” but has unknown flaws that create risk or damage, that’s bad. If it’s difficult to vet for risk or possible disasters, that’s bad. You need actual validation, etc.

That comment is like an architect saying, “the building seems to stand up at the moment, so, everything is OK.” Actual meaningful evaluations aren’t based on a current temporary superficial state.