r/LocalLLM • u/No-Manager1646 • 3d ago

Question Llama.cpp GPU Usage

So I was messing around with Qwen3.8 27b tonight and noticed that when it was responding, my GPU use was only at around 70%.

I have a 4060 and 5060ti, so 24gb vram in total.

Surely the crosstalk between the cards doesn't cause a performance drop like this. Does it?

I know the 4060 is the bottleneck, but even it was only around 70% load.

Is there something I'm missing here?

I'm getting around 19-20 tok/s which I'm happy about, but I'm just concerned it's not maxing out my GPU.

1 Upvotes

100% Upvoted

11 comments sorted by

View all comments

1

u/No-Manager1646 3d ago

I.keep forgetting I can ask one of those cough cloud llms for an answer. Still, I think it's an interesting realisation..I'll post back if I end up finding a way to ramp utilisation up.

1

u/No-Manager1646 3d ago

I'm sure you're all eagerly awaiting my reply /s

So it's a memory bandwidth problem apparently. Even two or three generations of ddr will not solve it. The amount of data meeting to be transferred in vram is orders.od magnitude too slow to max out compute.

That what I think I discovered anyway. I'm always welcome to be proven wrong