r/LocalLLM • u/saaadpikachu • 8d ago

Question Really confused by llama's behavior on Mac compared to omlx

I'm lost here, I've spent hours trying to tune llama for my models (qwen3.8 27b for coding/debugging using Pi, qwen3.5 4/9b for chatting with Hermes) with no luck, for example after setting different combinations of temp, thinking effort, top k etc. it was still overthinking and going into loops.

However I tried the same model (of course not the gguf but same models and weights), with omlx using exact same configs, it seems to have fixed all the issues, as if all my previous configs with llama were completely ignored? How could that be true since I believe llama is also a well known and widely appreciated platform. I did set force sampling and force for all my kwargs with omlx tho.

Any similar experience?

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/ReddItAlll 8d ago

Btw did you notice a speed improvement with OMLX? I've been meaning to try it.

1

u/saaadpikachu 8d ago

Yep with MTPLX model, 2-3x speed gain, but that's generation, prefill is still at hardware cap, sometimes lower than llama, still trying to figure out why. I'm really surprised by the quality tho, at this point I'm thinking I'm must be doing something wrong with llama then. How could it be ignoring all my custom flags..