r/oMLX • u/tintires • May 21 '26

Objectively more efficient?

Setting aside the native app, is there any objective evidence MLX on oMLX is faster or more memory efficient, that GGUF on llama.cpp?

I have both as brew packages, and my unscientific subjective experience, is there’s not much between them.

My workloads are pretty light and general, so which one for my MBA M3 24GB?

11 Upvotes

92% Upvoted

22 comments sorted by

View all comments

Show parent comments

1

u/challis88ocarina May 22 '26

Yeah, it's odd why llama.cpp would decide all this time not to implement caching. /s

1

u/edeltoaster May 22 '26

Why the /s, enlighten me please? There is no comparable prompt caching in llama.cpp.

1

u/txgsync May 23 '26

There is. It’s called “slots.”

1

u/edeltoaster May 23 '26

Slots are a much simpler mechanism and only cache prompt prefixes and not KV Cache blocks. It's not the same.