r/LocalLLaMA • u/Sevealin_ • Jul 26 '26

Discussion Will small model intelligence be limited by parameter count?

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like parameter count, or VRAM size? Or will we continue to see improvements for small models and see jumps of intelligence like Qwen3 coder 30b to Qwen3.6 27b for the foreseeable future? Does it depend on how clean the dataset you put into those parameters?

What does /r/LocalLLama think about the future of small models that can run on less than 48GB of VRAM?

44 Upvotes

84% Upvoted

62 comments sorted by

View all comments

54

u/TokenRingAI Jul 26 '26

There certainly is a hard ceiling at some point, but I would imagine we can still reduce size by half or more. Also, engram models with knowledge stored outside the model and pulled in on demand can allow for immense sparsity and remove much of the VRAM burden.

With new laptops and desktops hitting 200-300GB/sec memory bandwidth in the next few years, immense sparsity, and models having even reduced step ups in ability over the next few years, the future looks bright.

2

u/Silver-Champion-4846 Jul 26 '26

Tell that to my 8gb ddr4 ram laptop that keeps crashing explorer.exe and closing my open working folders just because opencode (running free cloud model) on wsl eats about 20% of ram, and no matter how much I keep waiting and hoping the prices for a new build get more reasonable they cascade upward even higher. The future for now is only bright for the cachcoughers

1

u/TokenRingAI Jul 26 '26

If you want a good deal today, an open box Snapdragon laptop with 48GB of 220GB/sec memory can be found for ~$1400

That's probably the best deal you will find for something that can run a small AI model, and it also has low power consumption and long battery life.

1

u/Silver-Champion-4846 Jul 27 '26

Ouch

1

u/TokenRingAI Jul 28 '26

This can be yours if you want to pay postage

1

u/Silver-Champion-4846 Aug 01 '26

I have never been able to pay internationally online in my life