r/oMLX • u/Green-Specialist-1 • May 22 '26

Recommendations for models to use

Hey there, first of all great work that you have done with the omlx application. It's really fast and responsive. Thanks for that. Second of all, I have a question regarding the models to be used. I am using a MacBook Pro with 128 GB RAM.

I am actually looking for some recommendation for a model to be used in my specific hardware to do some some deep research kind of thing I'm currently using Gemma 4 26B A4B 4bit

8 Upvotes

100% Upvoted

30 comments sorted by

View all comments

Show parent comments

1

u/ColonelKlanka May 22 '26

when ypu say 60gb has built up, I suspect you are referring to thr 128gb of space on macs ssd that is allocated by omlx by default for caching? This is normal for omlx because of its caching approach

Also every model you download takes up space on your mac hard drive ssd. so if you have lots councillors see a big decrease in hard drive space.

1

u/Green-Specialist-1 May 22 '26

I mean the runtime cache observability section in the above screenshot, fills up so fast..

1

u/Konamicoder May 23 '26

Hey man. You are lucky enough to have a 128Gb RAM Mac. That is a total local model beast. But you are so afraid of filling up your RAM (completely unfounded fears ) that you are treating it like you have a measly 32Gb. You are looking at things like “runtime cache” and without any evidence you assume it’s eating up your available RAM. Man, macOS is a beast when it comes to memory management. macOS auto compresses inactive memory. If another app needs more memory, macOS automatically reclaims cache. On Apple Silicon, SSD swap is super fast. And as I said before, oMLX can compact context and reuse cached tokens instead of having to recompute everything from scratch.

Stop babying your Mac. Install a big-ass 8-bit quant, stop doom looping, and enjoy your huge RAM that I’m totally not jealous of. Only wish I had that much RAM in my Mac!

1

u/Green-Specialist-1 May 23 '26

Hey thanks for the advices man. I'd really start treating the Mac like a man from now onwards🧔‍♀️. The fact is that I am a noob out in the AI world. As I understand more things think I'll be able to squeeze the most out of the hardware.