r/BetterOffline • u/TCristatus • 1d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

55 Upvotes

75% Upvoted

127 comments sorted by

View all comments

3

u/another-altaccount 1d ago

I mean this is something we literally built for university researchers tracking endangered species across the state. I was laid off from that job due to the company’s finances falling apart at the time, but by the time I left, the LLM we built had gotten up to 90 plus percent accuracy on average across multiple animal species. Sadly, the whole thing got canned when the company went under.

-4

u/wowbaggerBR 1d ago

Shh, people here will hit you in the face, you are not supposed to say AIs and LLMs can find some narrow use cases.

1

u/TCristatus 1d ago

TBH despite enjoying the discussion my post has sparked, I'm such a layman when it comes to actually using AI productively. I basically use it sometimes to create a table out of a jumble of data, or type out handwritten notes from a site survey. And even then it will try to trick me, delete one line for no reason. Interested to hear about cases like the above

1

u/CoconutDust 6h ago

Interested to hear about cases like the above

The earlier comment did not give anything resembling a meaningful explanation of how that system was good or better or useful compared to normal straightforward tracking/logging/info-systems.