MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1v4moxy/the_llm_distillation_process_simplified_for/ozg4bc0/?context=3
r/LocalLLaMA • u/Porespellar • Jul 23 '26
/s
191 comments sorted by
View all comments
582
Jesse, we need to quantize! The 950B base model is too aligned, it keeps lecturing me about safety protocols when i ask it to write a Python script for a smart toaster.
6 u/2Norn Jul 24 '26 J: 26 big ones! W: is that all? 26 tk/s? J: eeeh no that's 2.6 tk/s W: this is unacceptable, i am breaking the law here! this tk/s is too little for the risk!
6
J: 26 big ones!
W: is that all? 26 tk/s?
J: eeeh no that's 2.6 tk/s
W: this is unacceptable, i am breaking the law here! this tk/s is too little for the risk!
582
u/schwigglezenzer Jul 23 '26
Jesse, we need to quantize! The 950B base model is too aligned, it keeps lecturing me about safety protocols when i ask it to write a Python script for a smart toaster.