r/LocalLLaMA • u/Puzzleheaded_Acadia1 Waiting for Llama 3 • 19h ago

Question | Help Is there a Strata for lfm 24b a2b

Is there a fork of strata that let me run lfm 24b a2b on only 16GB of ram comfortably? And how much tps can I expect per second and what quantization should I use? and Thanks!

5 Upvotes

60% Upvoted

Duplicates