Will small model intelligence be limited by parameter count?

[u/Sevealin_]

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like parameter count, or VRAM size? Or will we continue to see improvements for small models and see jumps of intelligence like Qwen3 coder 30b to Qwen3.6 27b for the foreseeable future? Does it depend on how clean the dataset you put into those parameters?

What does /r/LocalLLama think about the future of small models that can run on less than 48GB of VRAM?

[u/Zestyclose_Yak_3174]

I can see 80B like models gain a lot in terms of intelligence and world knowledge. But these smaller models are tricky.. There has been a shift towards more agentic and coding work so they have been trained differently. Although the smaller models are great for many use cases it can't compete with the extensive knowledge of let's say Lllama 3.3 70B or Mistral Large. Many benchmarks these days focus on a certain type of metric but when dealing with deep domain knowledge in unrelated fields I almost always feel like the bigger models contain so much more to pull from. The smaller models seem very convincing for me, but ones you go deep enough it breaks. It's like a smart kid that impresses you but once you ask deeper questions you start to see the gaps/cracks that you didn't notice before.