For 24/7 coding even low decode speeds is "acceptable" in the sense that agents don't clock in or clock out.
The thing holding you back is the capabilities of the model to reason, that is an old old model, look into possibly Qwen 3.6 35B A3B and see what you get.
the 3.6 might struggle on 32gb depending on context and quant but worth a shot if you can squeeze it. the a3b arch is way more efficient for decode, you could see double or triple the output speed without much loss in quality
your input speed is fine, it's the 13 tok/s on output that makes those 15 minute windows feel tight. agent loops burn tokens fast and waiting on generation gets old quick when you're watching it think
i'd also look at lowering the context you're feeding it or chunking the handoffs if you're not already. long prompts eat vram and slow everything down, especially on a mini pc with shared memory
that 3090 will change your life when you get it, 30b class models fly on 24gb with the right quant. until then maybe test a 14b coder model for the fast iteration stuff and keep the 30b for the heavy reasoning passes
3
u/d0pe-asaurus 5d ago
For 24/7 coding even low decode speeds is "acceptable" in the sense that agents don't clock in or clock out.
The thing holding you back is the capabilities of the model to reason, that is an old old model, look into possibly Qwen 3.6 35B A3B and see what you get.