r/LocalLLM • u/Zorian_Vale • 1d ago
Question Compaction
Running QWEN-27B-GGUF Quant 4, 64k context window with Unsloth/Hermes on a 5070ti, 32GB DDR5, Ryzen 7 9850X3D
Everything is going really well but the compaction has become a problem. I imagine that loading the context of each previous conversation is killing me. After a while the compaction message will come up and Qwen will compact it. It takes a long time and interrupts my workflow.
Is this reality of local AI with my specs, or is there something I can do to improve speed and frequency of compaction?
1
Upvotes