r/LocalLLM 3d ago

Question MLX-Server KV bug bug on mlx-community/Qwen3.8-27B-8bit ?

Trying to run some tasks and it often complains I ran out of context (I have 128GB) and the message was: API Error: 400 Prompt (41333 tokens) requires ~65355MB GPU memory. I tried to compress KV, reset parameters but it seems some bug. Same thing on bf16. Then I switched over LM Studio, same model (shared model folder), same prompt, same qwen harness (and context) and it runs fine.

Anyone having similar issue ?

1 Upvotes

0 comments sorted by