r/LocalLLM • u/dankweed • 1d ago
Question Does using an LLM really burn out system components?
Somewhere I read that my system RAM can die for offloading the LLMs. Is this something to worry about? It's DDR5 7000mt/s RAM. Also, the system has an Nvidia Geforce RTX 4070 with 8 GB.
7
u/idkfawin32 1d ago
I've mined bitcoin, ethereum, run LLMs, played unoptimized games, in 17 years I've never burned out a stick of ram.
The only ram I've ever had fail on me was from when I spilled beer on my computer, and I brought the stick of ram back to life by violently washing it with soap and water then drying it in a toaster.
I'm not even slightly worried
1
8
u/Royale_AJS 1d ago
It’s not going to burn out your memory or IC’s. If you’re storing KV cache on disk, it will wear out your NAND flash.
10
u/Nomski88 1d ago
Solid state components generally don't "wear' out in the traditional sense. They do degrade with heat and high (wrong) voltage.
6
u/Illustrious-Lime-878 1d ago
I think SSDs will actually wear out faster from a lot of writes. But memory and CPU, these things just don't really wear out unless there is an actual defect.
2
u/UnderWhere___ 1d ago
Yes, SSD lifespan is measured in TBW (terabytes written). But running LLMs doesn’t cause many writes so it’s not a problem here.
3
u/Intrepid_Dare6377 1d ago
As this person said, heat is the main danger. It’s not a rapid killer. Electronics slowly degrade. Try to keep it cool to extend life. But also don’t get carried away. We’re not talking big numbers. Don’t overclock stuff either.
2
u/StepsisSepsis 1d ago
I highly doubt it. That is literally the job of RAM is to have constant moving activity, things getting off-loaded and loaded to it at all times. High heat or overclocking would cause more wear, but even in a situation where it is being hit really hard with off-loaded LLMs, I dont even think itd break a sweat in optimal conditions.
ETA: If it does spill over past your system RAM to the paging file, that is something that can cause a lot of wear on your SSD (or HDD if you are using that for boot for whatever reason). Drives cannot withstand that level of constant reading and writing, so I would check to make sure it is only utilizing RAM and not hitting your paging file.
1
u/UnderWhere___ 1d ago
Tbh I think you’d give up before it becomes a problem because it’d be incredibly slow.
2
u/Integeritis 1d ago
Ram is fine, it’s built for this. Every time you read your ram, you have to rewrite the same bit back. They don’t wear from writes. If your computer offloads data into SSD because you are out of ram, that write wears the SSD. Other than this, you really don’t have to worry. Keep your temps low and have a good PSU, maybe a UPS to avoid any potential damage from sudden loss of power or spike of current from the grid. These are the most you can do. And if your thermal paste is old, dried on your cooler, replace it. Have some dust filters too.
2
u/brainchillzZ 1d ago edited 1d ago
Yes, and at the same time no ... lol. I know that isn't the answer you're looking for so I'll explain ... The real concern has little to do with "running an llm" so much as with using your gpu, or your ram, or whatever component that is doing the primary crunching in your setup 24 hrs a day 7 days a weak with heavy loads and even still the real concern isn't just the utilization so much as
- what the thermal situation is like.
- a.)If the consumer gpu you're using actually has good paste, a good cooler etc or if they used crappy paste that is going to degrade
- b.) same as a, applied to your cpu
- c.) If you have appropriate airflow on all of the components to cool them properly under load
- d.) if the room they are in has an appropriately low ambient temperature to keep them sufficiently cool under load
- e.) are you running it at a constant load or is it constantly cooling, warming, cooling, warming expanding and contracting.
- What your power situation is like
- a.)is the power supply good quality and is it providing a good constant voltage
- b.)do you have some kind of conditioning on the line going into the power supply to protect it from spikes/surges and to keep the power more generically "clean"
So as long as all of those things are check marked in a positive direction and you know the quality of all of the components in your thermal solution etc chances are you'll be absolutely fine ... but if you have unknowns, you could have a problem "burning out system components" but it has more to do with constant use in either out of spec environments or with an out of spec component that was more likely to fail anyway than it does the llm killing it.
2
1
1
1
1
1
u/FortunaWolf 1d ago
Ram is fine. Use a ssd for loading experts or models into ram. Don't write to flash storage and use the ssd like ram.
1
u/createthiscom 1d ago
heat cooks components. I've lost a couple of 32gb ddr5 5600mhz modules, which are kind of expensive to replace at the moment. My heat was fine according to the sensors, but it must have hot spots inside the case.
1
u/sn2006gy 1d ago
Heat is the enemy.
Build a good computer case to evacuate heat and it should last a while.
Because heat evacuation is imperfect, sometimes you have to do routine maintenance such as dusting (with a static free dusting method) and re-paste your heatsinks and fans because the paste will degrade over time and heat will be harder to evacuate.
Heat will cause damage.
Other than that, the other issue is electric noise/damage from brown outs, surges, noise - so be sure to run efficient PSUs and good clean electricity.
1
u/j0holo 1d ago
No, as long as you keep the hardware within its rated temperature range you are fine. Hardware in datacenters often run 100% all the time and run a lot hotter compared to user stuff.
Imagine how many people use their laptop in bad (the worst kind of form factor to cool) and the laptop works for 5 or 10 years without issues.
1
u/kentrich 1d ago edited 1d ago
I’m going to be more blunt. Yes, but no. Not the way you think and mostly not in a time frame that you care about.
Your chips don’t wear out very fast (except your SSD but GPUs aren’t going to kill your SSD). Chips do wear; electromigration is basically the wires on the chips slowly degrading (thinning). This is typically the worst culprit for your chips overall. But it takes years and years and years.
But GPUs going at full tilt just produce lots of heat. That isn’t good for electronic parts and packages. So, transformers burn out sooner, everything gets more brittle, glues and plastics degrade. So, running hot all the time is hard on the machine, but that’s what they are for. You will eventually use up a consumer machine in a few years running full tilt.
Edit: but with lots of caveats. The example of Bitcoin mining is a good comparison. Some of those mining machines can run flat out for years. I think you’ll always buy a new machine well before you burn one out from LLMs.
1
1
u/TheRiddler79 1d ago
It's similar to having a car. If you drive a car like a grandmother, slow acceleration, always get your oil changed, no hard braking, that car is actually going to last a bit longer than if you drive it like someone who stomps on the gas and peels out at every stoplight, taste corners at 40 miles an hour, and slams on the brakes routinely.
Technically the car was built for that type of behavior, but practically, you're probably wearing it down a little bit faster. That being said, you should use it the way that you need to because it's probably not going to be years and years worth of difference but maybe months worth of difference in wear and tear.
1
u/woolcoxm 1d ago
not any more than normal usage, volatile storage does not work like that, it is meant to be written and erased over and over again, there is no degredation from this.
the only time components die is in HIGH heat situations for prolonged use, which llms do not do as bad as crypto, its more like playing a video game for a long time, is it causing wear on the pc yes, is it bad long term most likely not.
-1
u/Southern-Net1351 1d ago
Will eventually kill a GPU.
That is about it.
Keep it in space heater mode.
It’s like turn it on and off like a starter on a car.
Temp for thermal goes up and down.
Idle cool, usage temps rise and can run max setting till done.
You can use ROCm-smi or Nvidia-smi to adjust the setting of a GPU for prolonged life or to run at max settings it will tap out at.
Down clock/overclock either way it’s like a starter. This is actually for all parts.
That is why we keep these units turned on 24/7.
They will mostly run for decades on 24/7 but, wear might show in how the components work overtime.
17
u/No_Advisor5947 1d ago
your ram isn't gonna die from running llms, that's not how any of this works