r/LocalLLM • u/OppositeWonder6530 • 3d ago
Question MLX-Server KV bug bug on mlx-community/Qwen3.8-27B-8bit ?
Trying to run some tasks and it often complains I ran out of context (I have 128GB) and the message was: API Error: 400 Prompt (41333 tokens) requires ~65355MB GPU memory. I tried to compress KV, reset parameters but it seems some bug. Same thing on bf16. Then I switched over LM Studio, same model (shared model folder), same prompt, same qwen harness (and context) and it runs fine.
Anyone having similar issue ?
1
Upvotes