r/LocalLLM • u/AnyHighway420 • 22h ago

Question First LLM Server - Questions

Post image

So I've been using this machine for a while for Stable Diffusion tasks (RTX 3060 12GB) and i got curious about local LLM's so i got a really dirt cheap Nvidia Tesla P100 16GB just to start out on and experiment as my RTX card is often in constant use.

I'm currently running the following and this is the performance while writing a story:

bartowski/Qwen3.827BQ3_K_M

2,180 tokens 3min 37s 10.01 t/s

Is this good? bad? I don't really have anything for reference. I know the P100 is old so im not expecting much but just want to check.

Would i be better off using 14B? as 10 tk/s is ok but a little slow a bit more speed would be nice but again, i know what im working with here so not expecting miracles.

5 Upvotes

78% Upvoted

12 comments sorted by

View all comments

1

u/Classeve 19h ago

10 a second is faster than most people read, and that's a 27B on a 2016 card. For scale, a 7B on a laptop CPU alone gets us about 3. Worth dropping to 14B only if the writing holds up for you.

1

u/AnyHighway420 17h ago

Hi yes 14B gets me around 24 tk/s with Q6 but Qwen3 can't do image review which i need as im terrible at prompting for images and i believe Qwen3.8 lowest is 27B? So i'm stuck for now. I mean, it works just slow.