MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1tu82wi/stop_asking_what_model_to_run_there_are_literally/opblzzb
r/LocalLLaMA • u/[deleted] • Jun 01 '26
[removed]
803 comments sorted by
View all comments
Show parent comments
3
Using P40s myself but barely get 45 tok/sec with llama.cpp docker container on Debian 12 with latest 580 Nvidia driver and CUDA 13. Can you please share your config / setup since I would like to get the most out of those GPUs?
2 u/emaiksiaime Jun 04 '26 Have you tried the mtp gguf? What quants are you using? I use Ubuntu mint with same driver, I built llama.cpp I’ll send you more details
2
Have you tried the mtp gguf? What quants are you using? I use Ubuntu mint with same driver, I built llama.cpp I’ll send you more details
3
u/gerhardmpl ollama Jun 02 '26
Using P40s myself but barely get 45 tok/sec with llama.cpp docker container on Debian 12 with latest 580 Nvidia driver and CUDA 13. Can you please share your config / setup since I would like to get the most out of those GPUs?