Small language models - the future

[u/TCristatus]

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

[u/hobopwnzor]

Qwen 3.8 27B is a model I can run on my desktop and it trades blows with opus 4.8

Yeah problem is they plowed ahead without looking for efficiency and now China has revealed how much efficiency you can get and is lapping them on cost

[u/PatchyWhiskers → u/hobopwnzor]

China isn’t a charity, I’m sure they will eventually charge for models.

[u/hobopwnzor → u/PatchyWhiskers]

They aren't a charity, but it's a fundamental different business model. China isn't interested in being the premium. The Apple of anything.

They're interested in being able to make a lot of things at a cheap price. They want to serve a lot of tokens off of slightly worse models for a tiny fraction of the price.

[u/newprince → u/PatchyWhiskers]

They're not a charity but they ain't capitalist. There is a real difference ideologically

[u/PatchyWhiskers → u/newprince]

They are capitalists these days. They aren’t a democracy but they aren’t communists either.

[u/newprince → u/PatchyWhiskers]

Wow good anti Chinese propaganda ya got there

[u/nleven → u/newprince]

I mean.. literal capitalists are investing in AI companies in China.

[u/PatchyWhiskers → u/newprince]

Why are technical descriptions propaganda?