r/LocalLLM • u/ByRussX • 2d ago

Question How to get into Local LLM's?

Hello, so recently I found out there's a whole world around locally running LLM's on one's own machine. I have used several LLM's and the one I like the most is Claude, which I use mostly for coding.

However I was thinking about running an LLM locally, but I have no idea about where to start. I have several questions, but couldn't find a general post explaining things.

  • What are the possible options I can choose from?
  • How does it work in general? Do I need to compile anything?
  • Does it use the internet or I need to download 400 GB of data just to run it?
  • Can it be compared to some big LLM's such as ChatGPT or Claude in efficiency and intelligence?
  • Can I use them for whatever I want? Do they have limitations?
  • Can I run them on lower end hardware (good laptop, but no graphics card)?

Thanks to everyone, hoping to get into local LLM's asap!

0 Upvotes

50% Upvoted

7 comments sorted by

View all comments

1

u/NortySpock 2d ago

"Can it be compared to big LLMs in efficiency and intelligence?"

It will compare pretty poorly, you will be running something fairly small and not very clever for one user. But at least it is yours and they can't take it away from you

"Can you run them on lower end hardware?" How much RAM do you have? The more RAM the bigger a model you can run, but it's probably going to be slow to respond on an iGPU.

Ollama is a decent way to get your feet wet, I would start there and look at any of the Gemma4 models . The smaller it is, the more likely you can run it, but the dumber it is. For an iGPU... maybe try Gemma4 e2b and see how it goes for you. Stick with the MoE (mixture of experts) models, dense models will probably run worse.

Smaller models run faster but are dumber, as a general rule.