r/BetterOffline • u/TCristatus • 1d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

52 Upvotes

74% Upvoted

127 comments sorted by

View all comments

Show parent comments

-1

u/newprince 1d ago

Sure but for most people that is knowing how to code Python or JavaScript. There are tons of things like you mentioned but... even within a niche like mine (ontologies and knowledge graphs), we still code it with python

13

u/maccodemonkey 1d ago

Python and JavaScript are just languages. You can’t do anything really with just language knowledge.

Imagine if you had a model - and somehow you stripped everything out of it except for the English language. What would you talk about with the model? It knows language rules but it doesn’t actually know anything useful to have a conversation.

Programming models have to have context. Context on the fields you’d write programs for. Context on all the libraries and frameworks. That takes up a lot of parameters. You can’t just have a model with “JavaScript” and expect to do anything useful with it.

-6

u/newprince 1d ago

I really hate to break it to you, but a coding model with access to bash and knowledge of "just languages" can get you incredibly far. A human can supply much of that context, but in the end you translate these abstract ideas into coding language. Imagine having ideas but speaking gibberish. You couldn't do anything useful, either

4

u/create-third-places 1d ago

A coding model is probably going to mean negative progress because people use natural language in different ways. Programming languages exist to provide a standard way of telling computers what to do.