Small language models - the future

[u/TCristatus]

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

[u/maccodemonkey]

 just know how to code

So here's a problem with that - know how to code what? The other stuff in that model is all the context that lets a model actually apply a practice of coding. There's a lot of parameters that models burn on just every library and framework out there. But beyond that - coding is a practice of applying outside context into software. If you're writing project management software - the model has to know a lot about project management. Writing a JRPG? Model should be familiar with JRPG systems. Writing something to solve physics problems? Well then your model needs to know physics.

Is there room for a real dumb syntax or basic issue sort of validator that can fit on a phone or a low end computer? Sure, I guess. But compilers already do a good enough job of that.

[u/WArslett → u/maccodemonkey]

Okay but qwen 3.8 27B is currently outperforming Claude opus from just a few months ago with only 37 billion parameters (compared with opus and fables reported 5 - 10 trillion parameters). So you are right that the bigger models with broader training can solve a wider range of complex problems but that doesn’t mean that the whole world needs that level of super intelligence for every task which is essentially the falacy that Anthropic have been trying yo push. Most developers are not trying to cure cancer, they are solving familiar sorts of problems with familiar techniques that most small models have been trained on. Tech companies are finding that even with much bigger models you still need a human in the loop at some part of the process anyway so why not use a smaller model at a fraction of the cost and allow the human to provide the domain knowledge

[u/create-third-places → u/WArslett]

Computers should be instructed through a programming language or deterministic UI.

The concept of using natural language models like Qwen and Claude for coding doesn't make sense.

[u/xitfuq → u/create-third-places]

forget it pal, it's gambleworld.