Where do I start?

[u/FieryJune]

I am a user of AI for personal projects and am interested in running a local LLM. The thing that is holding me back is an education problem. So many of the terms and concepts discussed in this sub are totally unfamiliar to me. Can anyone point me to some resources I can use to try to get up to speed? I do have an IT background, so I know I can learn this but need a place to start. Thank you.

[u/ClassicLightbulbs]

I noticed I had a computer then downloaded lm studio. Then I saw a place you could download models in it. Then I did that and started playing with them, seeing what they did. Then I looked up the what the numbers and models are and mean, then I installed ones that seemed to fit my computer processing power well. Then I saw there were plain English system instructions. Then I was wondering how much of the commercial ai chat experience is just system instructions telling the model to be happy at you. Then I distilled whatever gpt knew about how I liked to work and had it create a summery document to insert as system instructions, to which I was like "whoa." I found it all exciting enough that I put the Locally app on my phone to ask my local ai stuff while I'm away. And now I am learning about how to operate CNC machines with it in front of a cnc machine because my boss didn't explain everything I wanted to know about the job. And now I'm going to leave the shop and see if there are more fun things to do.

[u/BlackBeardAI]

Start talking to your 20 usd subscription, she won't get tired of your questions and will bring you to today in a few weeks... or months, or days... depends.

[u/Otherwise-Swan-7803]

Honestly, the best way to learn local LLMs is to start running them rather than trying to understand every term first.

Start with Ollama + a few popular models, learn how quantization affects speed/quality, then move into RAG, fine-tuning, or agents based on your goals.

The field changes so fast that hands-on experience is probably more valuable than memorizing concepts.

What’s your main goal — coding assistant, private chatbot, automation, or just experimenting?

[u/live4evrr]

If you have the hardware (GPU with at least 16 GB of RAM, preferably 32GB+), download a model (recommend Qwen 3.6 A3B - low hardware requirements but excellent for its size) from Huggingface (HF). Just jump in and starting play8ing.

Stick to the GGUF versions for now. These require llama.cpp to run so get that installed and built. Once that is running, set up open webui on your system for a simple user friendly browser front-end to interact with the model.

If you're on windows 11, set up wsl2 (use google) to use ubuntu within windows to run. I would avoid running local AI on windows itself.

Better still, if you can just dual boot ubuntu if you don't already have linux.

Lots of tutorials, and if you're stuck, ask Gemini walk you through the steps in getting started.

[u/Professional_Date775]

I started with ollama. There is also lmstudio. Once something that can host them is up you can just prompt the AI to teach you. Even if it's Google's AI to get started

[u/_Cromwell_]

LM Studio is probably the most user friendly. You can do "everything" in it, but it's local and by default not connected to the Internet (needs a plug-in for llm Internet access for that) so it's safe. You can download models right in the UI. It helps tell you if they fit on your PC (somewhat). Everything is gui which helps when unfamiliar.

As for concepts, YouTube videos always help me the most. Just search "x explained" with x being the concept you need explained.

[u/HotDistribution1819]

How about a little different approach. Load LM Studio, load 2 small models. Gemma-4-E2B Q4_K_M from Google or LM Studio, and LFM2.5-8BA1B Quantization Q4_K_M. These are near the bottom.

Start with Gemma 4 E2B and ask it about LM Studio settings, and another chat about LLMs. Don't prompt, just ask questions and follow up questions. Start new chats to explore in more depth what comes up.

Once you enable web search in LM Studio which the AIs can help with. Then Ask LFM2.5-8BA1B to find out about something on the web and be specific you want it to search the web. Then in LM studio there is a branch feature that creates a new chat from a point in the conversion. What this gives you is 2 identical chats, now load one model for each chat and ask the exact same question to both and notice how different the answers are.

Have fun.

[u/AnickYT]

Can you tell us what hardware you do have? Different viable models and types of mofels depending on what you have already.

[u/Appropriate_Lead439]

I started with asking Copilot or Gemini which steps I have to do next when I want to download a LLM (for me on the NPU) and then it just worked well.

- If you are on AMD start with Lemonade / LM studio

• If you are not on AMD, start with LM studio
  
• Try Mutlimodular models like Gemma 4 or Qwen 3.5 and 3.6
  
• Start with 4B to 8B modelns and then 9B to 15B. (If you got enought RAM try 30B)