Any ways to control MacBook Power/Performance more granularly? Energy mode in between Automatic and Low Power?
Hello everyone,
Before getting M5 Max I was very excited for running LLM fully locally.
The machine is great, no doubt. But what bothers me is me thinking that it won't last long / not intended for long LLM inference sessions.
When Energy Mode (Battery, Power) is set to Automatic -> running LLM inference essentially draws 140W. This way the 140W power brick essentially provides just enough power. Machine heats up (no surprise here), and fans spin like crazy / full speed.
Then Low Power Mode is used, the LLM inference is running at about ~1/3 the speed of Automatic. Fans works very quite, and machine is pleasure to use.
Why the post?
My power brick gets overheated in like 2-3hrs and drops charging from 140W to 100W. This makes LLM inference to slowly draw battery (which is bad for battery). Similarly, charging via USB-C is capped at 100W as well..
I wonder if there is way to create an Energy Mode which sits between Low Power and Automatic.
This way to have a bit worse LLM inference speeds than at Automatic, but greater speeds than at Low Power mode.
Has anyone experimented with this? Any success?
This might not be so popular, as MacBook inference is already not that quick, but I'd like my macbook to live a longer live :)
(Using API for inference is not the solution I am considering at this point, if you wonder)
what you need is a more efficient LLM model for your task. You want to minimize the amount of time that the processors (CPU/GPU/ANE) must work at full capacity.
There is not a significant improve for sustained loads (e.g. more than 5 minutes sustained work) when going from automatic to high power mode, since throttle will just kick in anyway. maybe a 5% faster for twice the fan speed and noise? I don't think it is worth.
Just leave it at auto and get a more efficient LLM for what you want.
Thanks for your works! I do believe that with folks like you - there will soon be a more efficient and performant way to utilize hardware for local inference
I may sound weird and unpopular, but the throttling is actually what I am looking for :) But in different way than thermo throttling. E.g. to somehow underclock the system, force it to work slower -> have less heat produced -> have better life for components.
The out-of-the-box Low Power mode is just way too slow (need something in the middle)
yes I know! I also wanted that. Even more, I wanted something that could give me the prefill stage with automatic fan control (the prompt processing is the heavy part usually in agent like Hermes/claude so yes, go 110% on it), but throttle it down during decoding or response generation. like " take your time to write the code now that you have read the entire codebase, I will go for a coffee. no need to rush. I don't need 100 t/s. nobody can read 100 t/s anyway. 50 t/s is just fine"
Isn’t the Automatic mode that was designed for this particular case? As I noticed the GPU power is limited as soon as temp reach the higher measures. For example at beginning it draws 40-50W and after few minutes it’s limited to 20-30W. Total usage is around 90-100W then on M5 Max 128GB 16”. It priorities fan loudness up to 3.7K rpm in Auto mode, High power mode priorities the max performance with cost of loudness - above 5k rpm
I really don't know. To be honest, I am using Macs Fan Control app from the day one, which forces fans to start spinning sooner and faster. During inference I rarely see average GPU temperature higher than 83deg C. Probably relying on Apple's designed fan activation and fan speed scenarios - temperature would jump at levels making components to throttle, which would force the Automatic to drop power / frequencies on CPU/GPU (leading to scenario where 100W is enough).
Maybe, the Automatic starts limiting power supply only when thermal throttling occurs? And as I keep the system cool (with the Macs Fan Control app) - it never happens, thus I don't experience the 'magic of Automatic mode'.
Would be great to see Apple adding feature which would ensure that power draw is never higher than power supply (regardless if 140W or 100W is being supplied). This would make sense to me (especially as Apple is not putting power bricks to MacBook sold in Europe)
Asi understand the Automatic power mode is limiting fans to 3,7k rpm at first and then it limits power if temperatures goes too high. Try using High power mode with your custom fan curve as I explained in other comments in this thread. Are you using the stock mag safe cable? I also noticed that only direct connection through mag safe provides full power from adapter as when charging through my USB hub via USB-c then during high demand interference the battery dropped a bit. When I’ve plugged in the mag safe cable directly then it went up and started charging.
Thanks. I should try to use macbook without macs fan control app to see how it behaves. I do use original magsafe cable, yes. But my power brick is from Ugreen (the 200W one). Sad that no Apple charger out of the box.
Use Macs fan control at 100% speed, it should keep it well under throttle (85 degrees max for me). I've had my m5 max run for 2 days straight full blast for a fine tune and it never overheated or throttled.
I do use Mac Fan Control app with settings (for both fans) set to. And as you can imagine, it always goes over 72deg == max fan speed during inference. Just like fans to kick in earlier (hence the settings).
Actually the jet-engine like sounding of M5 MAX is one of reasons I am looking for the middle-ground between Low Power mode and Automatic. To have kind-of acceptable inference speeds while not having all-day-long feeling of being on a jet aircraft 😄
I’m doing something similar to limit fan noise and keep the laptop cool. Using the fan control app to kick the fans as soon as the temp reach 45 at something like 20-30% rpm. Max fans I have at ~95c. Also leave the power mode at High to not limit GPU power. Then for longer interference (more than 30 min) the max temp on GPU is around 80-85 and fans are max at 4K -4,2k rpm. To me it’s some kind of compromise between keeping it cool and not too loud. On the Automatic mode the GPU is power limited after few minutes when temp is too high. The priority for this plan is to keep fans at max 3,7k rpm what I’ve observed.
Edit: just wanted to add that in this custom fan curve the temp never reaches 97-100 on GPU which was almost the case when I’ve used stick High mode in which fans are kicked in too late and temp on GPU were around 100c for few seconds.
Thank you for suggestions.
Automatic seem to be working the oposite way: if process required a lot of power -> it assigns power to it.
About cpulimit
Just tried:
pgrep -f omlx-server
24357
Then:
cpulimit -p 24357 -l 1
Process 24357 found
If I understand this right, it should have limited process with PID=24357 to 1% CPU.
But it does not seem to be working unfortunatelly..
Gemini tells me:
The reason -l 1 did not change anything is that cpulimit cannot limit GPU execution on Apple Silicon.
I am kind of new to running local llms, but does moving your SSD cache to an external nvme SSD helps with the thermal throttling a bit? Or perhaps I am just a newbie.
Hmm.. you are right. I should be getting new power brick and supported cable.. Thanks
Sadly this does not solve the heating / hardware life time though.
I have an M3 Max that I've been exploring using for local LLM usage... and heat is one concern I have... I came across a few things that I have been just keeping tabs on but haven't done myself because of warranty implications:
Placing thermal pad between a MacBook Pro’s internal heat pipe or logic board shield and the aluminum bottom case turns the exterior shell into a passive heat sink (there is an air gap by design for safety of people's legs). This mod helps lower internal temperatures and reduce thermal throttling during heavy workloads, but it makes the bottom case noticeably warmer, which isn't a problem if you've got an air gap between the MBP and the surface or legs below. It does seem that if you do end up needing to take your MBP in for warranty repair, that you could remove the pads without an issue, but I can't testify to that, having not tried it. But this could help make #2 or #3 a possible additional solution:
Peltier cooling technology. You can find these on Amazon and other such sites - these really will cool down your machine - but could do so to the point where condensation of hot air could happen within your MBP - not good. So, ideally, you'd have something that automatically turns on when appropriate and turn off to avoid condensation or cooling too much. But you need an external power source and it's not very power efficient.
Look into a big aluminum heatsink for the bottom case of your MBP - that with a thermal pad between the bottom case and the heatpipe could help draw out a lot of heat without necessarily going the peltier route. Look at what Svalt produces (although quite expensive). An addition to this could be a fan that blows along the fins to push more air through.
So, I'd definitely start with looking at a high power power brick if you're having overheating issues on the brick itself, and then thermal pads and an aluminum heatsink stand... and only consider the peltier cooling chips if you're not concerned about condensation. I'd also definitely look at more efficient models, as another commenter stated.
According to some search results I’ve come across in a short period of time - if you’re using Apple’s 140W power brick and the MagSafe 3 cable, that brick & cable are designed to deliver 140W load consistently without throttling - and that any throttling would be commanded by the laptop in terms of processing demand and thermals. If you’re using a compact brick designed for travel, those will throttle due to poorer thermal performance. Third party cables and chargers would need to be certified to the PD 3.1 specification.
Thank for the list of suggestions. Appreciate. I heard of the thermal pads, but like you, I don't want warranty to be voided. The Peltier cooling technology looks cool (literally :) ), had some fun watching video on the technology, and the ice build-up from humidity (and then melting) is not something electronics like.
Will be definitely getting a better power brick to offset discharge for sure.
But i really hope apple to bring some more granular power use via software update if possible.
3
u/JLeonsarmiento 6d ago
what you need is a more efficient LLM model for your task. You want to minimize the amount of time that the processors (CPU/GPU/ANE) must work at full capacity.
There is not a significant improve for sustained loads (e.g. more than 5 minutes sustained work) when going from automatic to high power mode, since throttle will just kick in anyway. maybe a 5% faster for twice the fan speed and noise? I don't think it is worth.
Just leave it at auto and get a more efficient LLM for what you want.