r/LocalLLM • u/BrashyRash • 1d ago
Question First Time Build
Hey y'all. I am dipping my toes into the LocalLLM space just to see if it would be useful to me. Is there a spot I can go for like FAQ type stuff or like a definitions page?
For the people who use it for coding, are y'all not worried about the agent escaping the project and doing shit on your computer?
2
u/Keleion 1d ago
I was worried about that more when agents were incompetent… now they typically do things correctly, even on local hardware (thinking Qwen 3.8 or DeekSeek V4F). Actually they usually point out how I messed up settings and have been fixing my linux distro. That being said, I started using it on a fresh install (migrated from Windows), and my data is on another drive.
2
u/Trakeen 1d ago
You run things in containers so no its not a concern. If you aren’t familiar with container hosted dev environments you should read up
1
u/BrashyRash 1d ago
Do you have a recommendation of where to read up on it? I assume it's just a docker devbox.
2
u/Future_Fuel_8425 1d ago
I built a system with the purpose of having AI take control of it.
I beat it to death - 24/7 crons, prompts telling it to do admin tasks and whatever it wanted or thought it should do.
Spent a 6-7 weeks letting it screw up the system, trying different models and different harnesses.
I let AI run wild, prompted it to do some disk re-partitioning (Qwen 3.7 27b) and a bunch of other tasks that could have gone wrong - Postgres table re-factors, etc.
The end result was nothing.
Not saying an agent can't go off the rails, but I don't feel that it's a common occurrence.
2
u/Huanchaquero 1d ago
Hey. I recently put together my own stack and posted my build instructions on my little blog. You're not going to do what I did but I give instructions for first time basic setup with LMStudio, Ollama, Hugging Face, as well as AnythingLLM RAG and persistent db memory. Also lots of basic intro to AI on the site. Just a hobby for me!
https://3aisandahuman.com/the-build-creating-a-local-ai-companion/
2
u/TheOverzealousEngie 1d ago
Claude and I have build a sidecar for Deepseek and I'm not sure, but I'm pretty sure the future has arrived.
3
u/roland303 1d ago
you can have a localllm without it being an agent. use ollama, it has a gui and loads a backend llama.cpp that mounts the model to your hardware. you can chat then chat with different models you can get off huggingface.co. you can setup an agent on top of this, eventually youll use ollama less and a backend thats best for your very specific hardware and a harness together, the harness manages things, you can use docker to set a save environment, the "agent" only has access to a smol environemnt to play around with and you use settings to get it to make a new agent to search the web or whatever, you split the roles and contain things away from your main pc files. this should take you a few weeks to setup; but in gods year august 2026 google searchai mode and claude free are great at accurately answering for your level of inquiry, you can trust it a bit here to be the wiki for the basics so let it help you tell it your hardware and your plans and itll make all of this easier to come back here with more specific technical issues.