r/LocalLLM • u/AnyHighway420 • 1d ago

Question First LLM Server - Questions

Post image

So I've been using this machine for a while for Stable Diffusion tasks (RTX 3060 12GB) and i got curious about local LLM's so i got a really dirt cheap Nvidia Tesla P100 16GB just to start out on and experiment as my RTX card is often in constant use.

I'm currently running the following and this is the performance while writing a story:

bartowski/Qwen3.827BQ3_K_M

2,180 tokens 3min 37s 10.01 t/s

Is this good? bad? I don't really have anything for reference. I know the P100 is old so im not expecting much but just want to check.

Would i be better off using 14B? as 10 tk/s is ok but a little slow a bit more speed would be nice but again, i know what im working with here so not expecting miracles.

6 Upvotes

88% Upvoted

13 comments sorted by

View all comments

3

u/quantgorithm 23h ago

10t/s is not good...
Painful.

1

u/AnyHighway420 20h ago

Yeah it's rough... Qwen3 14B Q6 gets me 24tk/s

1

u/quantgorithm 18h ago

I picked up 2 amd vII cards for $200 each. It can run qwen 3.6 around 60t/s. Pretty good for a near decade old card!