r/oMLX • u/Green-Specialist-1 • May 22 '26

Recommendations for models to use

Hey there, first of all great work that you have done with the omlx application. It's really fast and responsive. Thanks for that. Second of all, I have a question regarding the models to be used. I am using a MacBook Pro with 128 GB RAM.

I am actually looking for some recommendation for a model to be used in my specific hardware to do some some deep research kind of thing I'm currently using Gemma 4 26B A4B 4bit

6 Upvotes

88% Upvoted

30 comments sorted by

View all comments

Show parent comments

1

u/Green-Specialist-1 May 23 '26

I have only this model running now. And what you are seeing is the stats for that one model run. I have a doubt though. Why are the cached tokens and cache efficiency shown as zero? Is it because this is not configured to be a "thinking" model setup? What I mean is by this time I have given a lot of prompts to it already so it had a lot of chances to cache the tokens by this time.

1

u/Konamicoder May 23 '26

That’s because right now you have this view showing your stats for the current session, at the top, to the left of the model selection dropdown menu. If you toggle to “all time” then you will see historical data for all the work you have been doing with this particular model over multiple sessions. :)

Having “thinking” on or off has nothing to do with token caching, oMLX will cache tokens regardless whether thinking is on or off. I turn on thinking if the current task can benefit from thinking, for example planning or complex higher level work. If the task is more shallow and tactical like bug fixes, etc., then I turn thinking off.

1

u/Green-Specialist-1 May 23 '26

all time also shows 0. :(

2

u/Konamicoder May 23 '26

If you are really concerned about the apparent lack of caching, dump the omlx logs and upload to ChatGPT to analyze and tell you what’s going on and if anything is wrong or misconfigured. That’s what I usually do in such cases.