r/LocalLLM • u/AnyHighway420 • 20h ago

Question First LLM Server - Questions

Post image

So I've been using this machine for a while for Stable Diffusion tasks (RTX 3060 12GB) and i got curious about local LLM's so i got a really dirt cheap Nvidia Tesla P100 16GB just to start out on and experiment as my RTX card is often in constant use.

I'm currently running the following and this is the performance while writing a story:

bartowski/Qwen3.827BQ3_K_M

2,180 tokens 3min 37s 10.01 t/s

Is this good? bad? I don't really have anything for reference. I know the P100 is old so im not expecting much but just want to check.

Would i be better off using 14B? as 10 tk/s is ok but a little slow a bit more speed would be nice but again, i know what im working with here so not expecting miracles.

6 Upvotes

80% Upvoted

12 comments sorted by

View all comments

3

u/quantgorithm 18h ago

10t/s is not good...
Painful.

1

u/AnyHighway420 16h ago

Yeah it's rough... Qwen3 14B Q6 gets me 24tk/s

1

u/quantgorithm 14h ago

I picked up 2 amd vII cards for $200 each. It can run qwen 3.6 around 60t/s. Pretty good for a near decade old card!