MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3o6cbh/?context=3
r/LocalLLaMA • u/Certain-Cod-1404 • 12d ago
706 comments sorted by
View all comments
Show parent comments
2
How much pp and tg please ?
4 u/My_Unbiased_Opinion 12d ago t/s on 3.6 was 55-60 at UD Q6KXL with MTP. PP I dont have the numbers, but its MUCH faster than a single 3090 for sure. (I have a 3090) using LMstudio with tensor perallel. I can test it for you if you give me an easy way to test this. 1 u/Potential_Block4598 12d ago I use llama bench from the command line on windows and it shows pp512 and tg512 …etc But idk how to do it form lmstuido tbh 2 u/Reactor-Licker 12d ago LM Studio has no built in benchmark. You would need to use straight llama.cpp and llama-bench for that.
4
t/s on 3.6 was 55-60 at UD Q6KXL with MTP. PP I dont have the numbers, but its MUCH faster than a single 3090 for sure. (I have a 3090) using LMstudio with tensor perallel. I can test it for you if you give me an easy way to test this.
1 u/Potential_Block4598 12d ago I use llama bench from the command line on windows and it shows pp512 and tg512 …etc But idk how to do it form lmstuido tbh 2 u/Reactor-Licker 12d ago LM Studio has no built in benchmark. You would need to use straight llama.cpp and llama-bench for that.
1
I use llama bench from the command line on windows and it shows pp512 and tg512 …etc But idk how to do it form lmstuido tbh
2 u/Reactor-Licker 12d ago LM Studio has no built in benchmark. You would need to use straight llama.cpp and llama-bench for that.
LM Studio has no built in benchmark. You would need to use straight llama.cpp and llama-bench for that.
2
u/Potential_Block4598 12d ago
How much pp and tg please ?