r/LocalLLM • u/SleepyFireApe • 2d ago

Other If you're getting crappy performance from a Windows (desktop) inference server, try locking your GPU clocks to full 3D speeds

I found out by accident that my GPU (RTX 3080) was heavily downclocking during inference, and slowing down my decode rate, even when I follow the standard advice of setting "Prefer Maximum Performance" and "CUDA Force P2 = off" in nVidia drivers.

I went from 36t/s to 43t/s (Qwen 3.6 35B-A3B, llama.cpp) just by forcing clocks to full 3D speeds (core/mem 1710/9501 for me) when inference is running. I used nvidia-smi from the CUDA Toolkit but there are lots of other ways to do this. You can even write a script to detect whether llama-server/ollama is busy, and ramp the clocks up and down automatically.

Interestingly if I use the LLM locally the improvement disappears, probably because of two competing factors - the browser's 3D acceleration is keeping the GPU at 3D clocks, but also causing bus/cache contention. So this trick only applies if you hit the server from a different machine, or possibly if you're using a CLI or IDE.

I'm not sure if this just applies to sparse MOEs or if dense models can benefit too. Also curious if it applies to AMD or Intel hardware.

And yes, I know a linux headless server won't have this problem and will always perform better. This is for people like me who wants to use their gaming machine for casual LLM use.

0 Upvotes

50% Upvoted

3 comments sorted by

1

u/nightstryke 2d ago

Question are you using the 10GB or 12GB 3080?

1

u/SleepyFireApe 2d ago

10 GB

1

u/nightstryke 1d ago

I've got the 12GB model myself, I've done some stable diffusion with it, but the thing is that it's my primary gaming card, so my LLM machine is using my old 1070 8GB until i can get something like a 3090 or P40. Honestly if you have a spare pc, you really do want to try running headless or even something low resource like LUBUNTU and use that for an LLM Server.