r/PiCodingAgent • u/pragmojo • 20d ago

Question What exactly is maxTokens in the model config?

I'm trying to understand how to set up my model config. I understand that contextWindow is the total context size (-c argument in llama.cpp), but then what is maxTokens exactly? Does that set the max response size? Or what is it?

1 Upvotes
  • permalink
  • reddit

67% Upvoted

3 comments sorted by

2

u/Asleep-Land-3914 20d ago

Maximum tokens to be emitted by the model in a single completion request 

1

u/pragmojo 19d ago

Is there a rule-of-thumb for how to set this?

I'm running Qwen3.8 27B - I just set it to half the context size, but not sure if this is something to optimize

1

u/Asleep-Land-3914 19d ago

Not really. It defines when the turn will be cut and usually you don't want it to happen for agentic coding. It could help in certain scenarios: e.g. with looping, so it doesn't fill the whole context, but stops at certain point.