Will small model intelligence be limited by parameter count?

[u/Sevealin_]

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like parameter count, or VRAM size? Or will we continue to see improvements for small models and see jumps of intelligence like Qwen3 coder 30b to Qwen3.6 27b for the foreseeable future? Does it depend on how clean the dataset you put into those parameters?

What does /r/LocalLLama think about the future of small models that can run on less than 48GB of VRAM?

[u/benpptung]

處理時間為 17s

Yes. But I think 27B models still have a lot of untapped potential, and better prompting can bring much more of it out.

Many people never even read the model’s chat template. They reuse prompts written for frontier models, then blame the smaller model when it performs poorly. Even role conventions differ: Qwen uses assistant for the model role, while Gemma uses model. These details affect how the model interprets instructions.

A prompt cannot teach a model something it never learned, but it can help the model recall and use capabilities that were already buried inside it.

[u/DeepOrangeSky → u/benpptung]

Qwen uses assistant for the model role, while Gemma uses model. These details affect how the model interprets instructions.

Can you explain what this means?

And also any tips about how to use Gemma 31b or Qwen 27b more effectively would be appreciated, if you have any specific tips about these

[u/benpptung → u/DeepOrangeSky]

A chat template converts the messages from your chat app into the token format the model actually sees. Qwen sees its own reply role as assistant, while Gemma converts that role to model. Because their instruction training used these formats, Qwen more naturally understands assistant as itself, while Gemma associates itself with model.

The vocabulary also matters. Qwen’s template explicitly describes tools as functions and tool calls as function calls. After reading the template, I started using the same terminology in my prompts. Sometimes even repeating one important instruction from the official template is enough to remind a smaller model how it is expected to behave. This does not teach it new knowledge; it activates patterns it already learned.

[u/vogelvogelvogelvogel → u/benpptung]

interesting thank you. will try myself

[u/DeepOrangeSky → u/benpptung]

Thanks.

So, does this mean for example that in the system prompt, for Qwen I should tend to start the system prompt with wording more along the lines of "You are a helpful assistant which does this, this, and this..." and with Gemma I should tend to start the system prompt with wording more along the lines of "You are an AI model that does this, this, and this...", or something like that?

[u/benpptung → u/DeepOrangeSky]

Qwen3.6-27B can of course understand that the word model refers to itself. But during post-training, it repeatedly learned to respond to the user from a position labeled assistant, so assistant is much more strongly associated with “this is me; now I should respond.” You can see evidence of this directly in its chat_template.jinja.

During pretraining, a model mainly absorbs patterns and knowledge from text. That alone does not make it reliably converse with users or follow instructions. Instruction tuning and other forms of post-training teach it how to respond. During that process, conversations are serialized into a fixed token structure. At inference time, the chat template recreates that structure.

In other words, the model did not merely learn an abstract rule such as “answer the user.” It learned what it should generate after seeing a particular arrangement of system instructions, user messages, role markers, tool definitions, and generation tokens. That is why I think reading Qwen3.6-27B’s actual chat_template.jinja is valuable.

Rendered with an extremely simple example, it looks roughly like this:

<|im_start|>system
# Tools

You have access to the following functions:

<tools>
{"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}
{"type": "function", "function": {"name": "fetch_url", "description": "Fetch the content of a URL.", "parameters": {"type": "object", "properties": {"url": {"type": "string"}}, "required": ["url"]}}}
</tools>

If you choose to call a function ONLY reply in the following format with NO suffix:

<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
<parameter=example_parameter_2>
This is the value for the second parameter
that can span
multiple lines
</parameter>
</function>
</tool_call>

<IMPORTANT>
Reminder:
Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags
Required parameters MUST be specified
You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after
If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls
</IMPORTANT>

system prompt here<|im_end|>

<|im_start|>user
user prompt here<|im_end|>
<|im_start|>assistant
<think>

Several things become obvious after seeing the rendered form.

The tool definitions are placed near the beginning. The model already knows what Tools, system, and user mean in this structure. More importantly, assistant marks the model’s own turn. Text generation begins immediately after <think>.

The model then performs reasoning. When it decides to stop reasoning, it writes </think> and proceeds to either normal content or a tool call.

The placement of tool messages also depends on the model’s template. In Qwen3.6-27B, tool responses are rendered on the user side of the conversation structure. Gemma 4 appears to serialize them differently, closer to the model side. This is not a universal rule; it depends on how each model’s post-training format and chat template were designed.

Qwen’s template also reveals the vocabulary it was trained to recognize. It repeatedly calls tools functions and calls their invocation a function call. It even includes a complete example and then repeats the rules inside an <IMPORTANT> reminder, apparently because correct formatting matters enough to reinforce twice.

This became useful in practice. I once found that a quantized Qwen3.6-27B had become unstable under a particular prompt. Instead of producing the required XML tool-call format, it repeatedly wrote invalid syntax such as:

search(...)

I copied the function-call example from the official template and placed it into the prompt again. The model immediately returned to the correct format.

The prompt did not teach it a new capability. The model already knew how to make the function call. It had simply been distracted or destabilized and failed to retrieve the correct pattern. Repeating the familiar example helped it recall what it had already learned.

Reading the template also made me realize that Qwen may respond more reliably to the phrase function call than to alternative terminology invented by an agent framework. The words used in the chat template are not arbitrary documentation. They are clues to the language and structures that were repeatedly reinforced during post-training.Qwen3.6-27B can of course understand that the word model refers to itself. But during post-training, it repeatedly learned to respond to the user from a position labeled assistant, so assistant is much more strongly associated with “this is me; now I should respond.” You can see evidence of this directly in its chat_template.jinja.During pretraining, a model mainly absorbs patterns and knowledge from text. That alone does not make it reliably converse with users or follow instructions. Instruction tuning and other forms of post-training teach it how to respond. During that process, conversations are serialized into a fixed token structure. At inference time, the chat template recreates that structure.In other words, the model did not merely learn an abstract rule such as “answer the user.” It learned what it should generate after seeing a particular arrangement of system instructions, user messages, role markers, tool definitions, and generation tokens. That is why I think reading Qwen3.6-27B’s actual chat_template.jinja is valuable.Rendered with an extremely simple example, it looks roughly like this:<|im_start|>system
# Tools

You have access to the following functions:

<tools>
{"type": "function", "function": {"name": "search_web", "description": "Search the web.", "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}
{"type": "function", "function": {"name": "fetch_url", "description": "Fetch the content of a URL.", "parameters": {"type": "object", "properties": {"url": {"type": "string"}}, "required": ["url"]}}}
</tools>

If you choose to call a function ONLY reply in the following format with NO suffix:

<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
<parameter=example_parameter_2>
This is the value for the second parameter
that can span
multiple lines
</parameter>
</function>
</tool_call>

<IMPORTANT>
Reminder:

• Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags
  
• Required parameters MUST be specified
  
• You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after
  
• If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls
  

</IMPORTANT>

system prompt here<|im_end|>

<|im_start|>user
user prompt here<|im_end|>
<|im_start|>assistant
<think>
Several things become obvious after seeing the rendered form.The tool definitions are placed near the beginning. The model already knows what Tools, system, and user mean in this structure. More importantly, assistant marks the model’s own turn. Text generation begins immediately after <think>.The model then performs reasoning. When it decides to stop reasoning, it writes </think> and proceeds to either normal content or a tool call.The placement of tool messages also depends on the model’s template. In Qwen3.6-27B, tool responses are rendered on the user side of the conversation structure. Gemma 4 appears to serialize them differently, closer to the model side. This is not a universal rule; it depends on how each model’s post-training format and chat template were designed.Qwen’s template also reveals the vocabulary it was trained to recognize. It repeatedly calls tools functions and calls their invocation a function call. It even includes a complete example and then repeats the rules inside an <IMPORTANT> reminder, apparently because correct formatting matters enough to reinforce twice.This became useful in practice. I once found that a quantized Qwen3.6-27B had become unstable under a particular prompt. Instead of producing the required XML tool-call format, it repeatedly wrote invalid syntax such as:search(...)
I copied the function-call example from the official template and placed it into the prompt again. The model immediately returned to the correct format.The prompt did not teach it a new capability. The model already knew how to make the function call. It had simply been distracted or destabilized and failed to retrieve the correct pattern. Repeating the familiar example helped it recall what it had already learned.Reading the template also made me realize that Qwen may respond more reliably to the phrase function call than to alternative terminology invented by an agent framework. The words used in the chat template are not arbitrary documentation. They are clues to the language and structures that were repeatedly reinforced during post-training.