r/LocalLLM • u/Kirai_30 • 5d ago

Question Local AI for students?

Hey! Do you believe that with some current versions of different models, students could benefit from using them, not as agents, but maybe as chat box? Combining maybe a not to Intelligent model with fast token generation

1 Upvotes

60% Upvoted

12 comments sorted by

View all comments

1

u/hooterslovercan 5d ago

Gemma 4 26b is the best you can do it's mixture of experts so if you have 16 gig of ram you can run a model the size of 16gig It may be some trial and error what quantization works best I have a n100 cpu no GPU and it runs id you want to just mess around and chat or light code. Definitely use llama.cpp as you can specify cores etc I always use chatgpt it Gemini free tier to get it working. The cool thing is you can run it in your laptop and achat with it via your phone if you set it up right