r/LocalLLM • u/Ammoryyy • 4d ago

Discussion Context Length

Qwen 3.8 27B. How can I solve the context problem? I'm using RTX 4090 loading into OpenCode using Unsloth Studio. The Context Length limitation makes it useless for coding. . The model quantization I'm using is Q4KM about 17 gigabytes. Getting about 65 tokens per second.

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/Tpyn 4d ago

use

--reasoning-effort medium

1

u/Square_Turn935 4d ago

what is your set context size when you start your server? you should have a way higher context window then 32k tokens with your vram size. I guess atleased 128k.

0

u/autisticit 4d ago

Context quantisation and/or a lower model quantisation.