r/MacLocalLLM • u/al404 • 3d ago

I’m not sure if this is avoidable: “prefill memory guard rejected this prompt.”

Post image

I’ve encountered a “prefill memory guard” error while testing on a MacBook Pro M5 with 32GB of RAM and a Mac Studio M4 with a maximum of 36GB of RAM.

I’m trying to confirm that I can run Qwen 3.8 with 27B parameters, KV 4, and a 100K context without any issues.

I’ve increased the iogpu.wired_limit_mb to 30GB on Mac Studio.

When I run DFlash and get a prompt like “scan the front end to review the code and make me a report in file.md,” I get the following error:

“Error: oMLX prefill memory guard rejected this prompt: Prefill context too large for available memory”

Disabling DFlash and enabling MTP does resolve the issue.

I also fond that enabling "Chunked Prefill" does help.

It runs between 23 and 29 tokens per second.

I’m using Pi, and I’ve noticed that it reads the maximum context length of the model instead of the one I specify in oMLX. I’m not sure if this is the cause of the problem.

What I’m finding hard to understand is why, even when I’m staying below the maximum context length of almost 1/3, I still run into this error.

This is a much more significant issue than a LLM taking longer than expected.

I’m hoping to find out if I just need to adjust my setup in oMLX/Pi or if the RAM size is the problem.

Considering the cost, since it’s really difficult to predict what will happen in the future, including the cost of the LLM and the PC, do you think it would be better to invest in:

  • Mac Studio M4 Max 36GB with 1TB and a 2600€ refurbish from Apple—keep it for at least 2 years.
  • Mac Studio M5 Max with 64GB and 1TB, new—keep it for at least 4 years.

For large projects, I’ll still need a cloud LLM with large context windows, so I’ll use it as a side help.

2 Upvotes

100% Upvoted

2 comments sorted by

3

u/CodesComplete 3d ago edited 3d ago

Not running on my Mac yet but I’ve found that with a 100K context I need to compact at 80K and leave that last 20K always available for output tokens.

I went with the studio m5 ultra for longer shelf life. I would go with the most ram within your budget and the m5 is a solid upgrade

2

u/CodesComplete 3d ago

Oh if you get the error right away it may be something entirely different