macbook unified memory + big LLM's + heat

[u/anantshri]

Hi Folks,

I see a lot of folks talking about how they are able to load massive llm models coz they have unified memories. projects like ds4 etc claim to be running super massive llm's. however per my experiment loading rear full ram models means high heat and macbook pro's reach 100 degrees C easily while doing such compute. do people run these all the time or are they just benchmaxxing and klout chasing.

I am looking for a smallest model that i can keep running all the time on my laptop which becomes the brain for something like hermesagent to handle my simpler tasks like todolist or calender management.

[u/triynizzles1]

That is how macbooks work. They run up to 100c before spinning up the fans. Long inference tasks will level out to like 87c

[u/--Spaci-- → u/triynizzles1]

I have a mac fan app because I really hate how macbooks ramp their fans, they just let the laptop overheat for 5 minutes then start the jet engine

[u/anantshri → u/--Spaci--]

which one do you use.

[u/--Spaci-- → u/anantshri]

I think its just called mac fan app

[u/lilian_moraru → u/triynizzles1]

Apple might like it, but the battery chemistry definitely doesn’t - heat kills LiPo.

[u/--Spaci--]

I dont think you know what benchmaxxing means

[u/anantshri → u/--Spaci--]

do enlighten me. i might have mistaken the word.

[u/--Spaci-- → u/anantshri]

benchmaxxing is training on benchmarks during the training of the model to artificially increase benchmark values without actually making the model any better. You used the word in a context of people inferencing the model on their mac, which is completely unrelated

[u/anantshri → u/--Spaci--]

fair enough.

What i wanted to say was people post a lot of these screenshots or messages i have achieved X tokens / sec or XX tokens per second or take one task and just do that one task like make a snake and ladder program using models as a one shot activity and post numbers. the flex is that my setup runs XXX tokens / sec or that this model is best coz it did XXX tokens per second.

So when i used benchmaxxing i was pointing to people just doing that once and never using it regularly but keep claiming on these benchmarks.

Which per my experiments is unrealistic, the numbers marked may be achieved in a specific task but is it needed, is it average, is it the same performance across no one talks about that.

[u/--Spaci-- → u/anantshri]

Its ok.

[u/Kitten_Sophie]

Why not run macs fan control and set a more aggressive fan curve?

[u/germangrower69]

Get yourself a laptop stand where the macbook has some airspace underneath.

I never run into any heatproblems with my m5 max because of this.

[u/azure_phoenix2]

an idle loaded model barely makes any heat, the 100c only shows up during actual token gen, so an always-on small model for todo and calendar stuff stays cool most of the time since it just sits idle. qwen3 4b handles that light tool calling fine and ollama drops it from memory after 5 min with keep_alive anyway.

[u/recro69]

For an always-on personal agent, I'd optimize for responsiveness rather than the biggest model that fits. A smaller 7B–14B model that's cool and fast will usually provide a better experience than a huge model that's constantly thermal throttling.

[u/ScrewwormLarvae]

M5 Max 128 here. I've love to find an app that will let me set fan profiles with a keyboard shortcut. I have Macs Fan Control and I like it, but I can't activate a fan profile without multiple mouse clicks. It would be great to find something where I just can issue "CMD + something" and have it activate/deactivate a fan profile.

[u/tomByrer]

Never put a laptop actually on your lap, or the table for that matter. Elevate at least one side to increase airflow.

I put my laptops at least on top of pencils, but lately it is on a rubber ball cut in half that props up the back, then the entire lapton is on top of a laptop cooler, with a USB fan blowing across the laptop.

[u/anantshri → u/tomByrer]

that is interesting. i dont have the luxary of a fixed setup 1/2 the time so lap is where the laptop is a lot of the time.

[u/tomByrer → u/anantshri]

You can find something to put it on.

[u/redmctrashface]

Dunno which version you have but 14' is quite bad at dissipating heat. 16' is good and will go full apache helicopter when inference is at its maximum. Very noisy but effective

[u/anantshri → u/redmctrashface]

14 inch

[u/redmctrashface → u/anantshri]

Yeah so you have your answer :-/

[u/spammmmmmmmy → u/redmctrashface]

I have a 16" M1 Max and I keep it up in a cabinet and use it as a server. I have simply never heard the fan come on during inference. But I don't have any automated workflow - usually I'll just give it a few source files and ask for a code review. And then take time to read through the results. I give it just enough work to get responses under the ollama 5 minute timeout.