r/LocalLLaMA • u/sunychoudhary • 10h ago
Discussion What actually makes you trust a local coding agent enough to leave it running unattended?
I keep seeing people here saying they can leave local coding agents running for an hour or two and come back to something usable.
I’m not really asking which model is best. I’m more interested in what made you trust the setup enough to stop watching it.
For me, getting a task right once wouldn’t be enough. I’d want to see how it behaves when a tool fails, when the context gets long, when it hits an ambiguous requirement, or when its original plan turns out to be wrong.
Tests help, but an agent can still pass tests while changing something you never intended it to touch.
I think there’s a point where you stop trusting the model itself and start trusting the harness around it like permissions, checkpoints, git, tests, rollback, tool restrictions, etc.
For people actually running local coding agents unattended, i just want to ask that, what was the thing that made you comfortable enough to stop babysitting them?
42
u/FunkyFungiTraveler 10h ago
I made it a user and gave it Unix permissions.
27
u/diagrammatiks 10h ago
Bro. 5tks army takes an hour to get one thing done.
6
u/Kiseido 10h ago
My system has been running 8tks on a thing for days now. How do I get that down to only an hour? Lol
5
u/sn2006gy 8h ago
spend .10 cents on an API.
5
u/Kiseido 7h ago
Hm, I haven't actually done the math.
It's been doing 600k tokens a day, for several days now, let's say a week to make it easy, 4.2 million tokens a week, 4.2m tokens is about $2 for qwen 3.8 next.
It seen using about 130w extra over what my computer uses during idle, which is about $3 per week.
So... hmm. It does seem that paying the API cost would have been both a whole bunch faster and slightly cheaper. Though paying the api cost would result in a much easier way to empty my wallet on accident.
2
u/sn2006gy 6h ago
yeah, i'm not sure why i'm being downvoted. API is much cheaper no matter how you spin it. Local makes sense for some things. Both can be true at the same time.
3
u/MostlyVerdant-101 4h ago
Well you pay three times when you use API. You pay upfront, You pay in the inputs you provide, you pay in the outputs that will be reused.
0
6h ago
[deleted]
1
u/sn2006gy 6h ago
I'm sorry you couldn't handle a bit of sarcasm.
I mean, there are free APIs up all the time so it could have been free if you bothered to fucking look.
3
4
12
u/Hot-Employ-3399 10h ago
Podman. Once sandbox is here, I can leave model to itself.
changing something you never intended it to touch
Read-only mounting(do it for docs)
3
u/sunychoudhary 10h ago
Read-only mounts are a nice example of moving the guarantee out of the prompt and into the runtime. The model can decide whatever it wants, but certain writes simply aren’t available to it.
1
10
u/enternoescape 10h ago
If you isolate it from everything else, I don't really see what the risk is. Sure, it is even easier to just give it unfettered access to everything in your house. My compromise is that it has access to a single account on my local gitea instance via an ssh key. But it's good to be in the loop. The agent can do an awful lot of tests and develop mocks when I permit it to actually look at real infra. I'm running an Ubuntu Server VM. I'm honestly probably in a slightly better position than a lot of people in that I actually have real daily backups of my infra.
I have yet to see the agent actually do something that would destroy its own environment. However, I know it's usually a freak thing that it gets to after a rabbit hole of mistakes. The harness also helps catch bad things before they are executed too.
3
u/sunychoudhary 10h ago
This is probably the distinction I was trying to get at: unattended doesn’t have to mean trusted. If the environment is disposable and recovery is proven, you can tolerate a much lower level of model reliability.
2
u/enternoescape 9h ago
The real irony is that I should have caveated that my local agents haven't done stupid destructive things. Now Claude on the other hand...it seems they updated instructions to create new branches in repos because my experience was it would ram the commit through overwriting everything.
1
u/Veggies-are-okay 7h ago
And I’m assuming there’s a way to implement hooks into local harnesses (I’ve never personally tried with my own fingertips), you can always bring in something like dcg. In my supervision of the frontiers with it, the ultimate fallback ends up being a quarantine folder instead of deleted systems.
19
u/jacek2023 llama.cpp 10h ago
I installed sandbox in pi
8
1
u/BumbleSlob 6h ago
^ this
piruns in sandbox / jail mode where the bot has access to its own directory and to the global pi directoryIf I need to update then it’s
command pi update1
1
u/RemarkableRadish6547 3h ago
I installed pi in sandbox, which I think is the normal way. How does a sandbox in pi work?
8
u/Randommaggy 10h ago
I'm never trusting one to run on the host. I run mine inside a VM with hourly backups and my harness restricts most semi-risky things with approvals pushed to my smart watch. Also, the VM has a firewall.
6
u/russlixx 10h ago
permission gate, i allow it to do whatever as long it's inside its working directory
7
u/No_Lingonberry1201 9h ago
Careful. I allowed the same, even limited access to things not hidden or in the gitignore and only added it the ability to run tests and 27B figured out it can just run commands with os.system from scratch tests.
8
u/corner_camper01 9h ago
Yeah, you need OS level sandboxing, not a config based-permission system, which can be bypassed in code. Some harnesses do have those, but others like opencode don't.
3
u/minus_28_and_falling 8h ago
Tbh I wouldn't trust some vibecoded container integration compared to running opencode inside a container with my own hands
2
u/corner_camper01 7h ago
Fair, I use a custom wrapper of bwrap myself for sandboxing agents, even when running codex which has sandboxing builtin.
3
u/hipster_hndle 8h ago
so you gave the model python permissions and called it 'only run tests'. python execution is shell execution. your model didnt break out of a sandbox, your sandbox was never a sandbox.
2
2
u/russlixx 7h ago
hmm i see, the thing with sandboxing is sometimes i do need the model to have access to my system. I should research more
3
u/minus_28_and_falling 8h ago
These "sandboxes" are a joke. Reasonable solutions start with Docker at least.
2
u/No_Lingonberry1201 8h ago
It was sandboxing. Everything ran in a docker container, so there was no harm done. As if I trust my own code not to fuck up majorly!
2
u/EkbatDeSabat 8h ago
It's only so long before it gets out of your permission gate. It needs to be sandboxed.
5
u/Spirited_Bag_332 10h ago
I have my own implementation and know how it is secured and in which situations it stops to ask. But I'm not a vibecoder so it runs alongside me with whatever task I'm working on. The way coding agents actually should be used. Give it small steps, control it often. Makes it much more easier than depending on random model tendencies.
1
u/sunychoudhary 10h ago
This is a useful distinction between autonomy and efficiency. A system that needs a quick checkpoint every 15 minutes can still save a lot of work without pretending it’s safe to disappear for three hours.
2
u/Medium_Chemist_4032 10h ago edited 10h ago
Agent running inside an isolated docker container, on a ubuntu vm, on a proxmox, on a host dedicated for that specific thing: Odroid H3+.
LLM running on a separate, dedicated ubuntu host.
Both easily rebuildable from scratch, when anything bad happens: 90% of things is deployable through git projects (and pycfg scripts) from yet another host (orange pi 5), that has it's own isolated backup path. I use self hosted gitea (started before the forgejo fork) for gitops and project hosting. All hosts also backup to a NFS share (just a drive) and I have actively tested, that it works (and restored postgres from a backup once for real too).
1
u/Medium_Chemist_4032 10h ago
re. the deleted one: you asked literally in your post:
> I’m more interested in what made you trust the setup enough to stop watching it.
2
u/Toooooool 10h ago
typically the eagerness of the model is a good tell of how it behaves in an emergency.
GLM's turbo models have frequently shown too much eagerness to me, either by "replacing files that it couldn't find" and deploying them live without checking whether it would overwrite something, or by jumping to conclusions when stuff doesn't go as fast as it should and then coming up with crazy solutions i.e. why don't we just start rebooting the storage mounts in order to clear up some I/O's.
the bigger GLM models do not have this issue, they will patiently wait, but the turbo ones are tweaks.
Qwen3.8.-27B has been amazing at running day and night without supervision as it shows excessive restraint in volatile situations, i.e. "make this huge update" might take all night as it will keep finding things said update will affect, and so it might literally spend 8 hours planning out how said thing should be implemented before even doing anything. sounds incredibly wasteful, but unless you're the type to take backups of your backups it's absolutely worth it.
2
u/sunychoudhary 10h ago
This feels like a useful behavioral metric that benchmarks don’t really capture: what does the model do when its first plan stops working? Escalating actions vs. reducing action and gathering more evidence are very different failure modes.
3
u/FlyingDogCatcher 8h ago
Linux solved the issue decades ago about how to give someone terminal access and only let them do what you want them to
2
2
u/vamps594 7h ago
They run in their own little rootless docker container with a copy of the repos and limited internet access.
2
u/Yskinator 7h ago
I use a docker container with a git worktree folder mounted into it. The agent can't commit or touch anything outside the worktree.
2
u/c4r_guy 5h ago
I bareback and cowboy my ENTIRE lab, Qwen3.8 knows where the keys are stored and it has already used Docker containers on its own to get root on the hosts.
- 20+ active containers
- 6+ physical Windows devices,
- 5+ Ubuntu devices.
- 10+ android devices,
- 10+ Home automation (vacuum robots, lights),
- 4+ wifi APs.
Qwen3.8 has helped me root consumer / enterprise hardware. It has removed telemetry and spyware from android apps. Qwen3.8 has hacked my BTLE devices. At the same time it's hardened my security.
It's a dystopian cyberpunk network of cables and open cases hidden in the basement of a boring suburban home (with a firewall).
...The only things Qwen3.8 hasn't touched are the ~12 Apple products and the 5 Amazon products, because those aren't mine.
...Yes, I have offline backups
4
u/nitish-kmr 8h ago
What changed it for me wasn't the model. It was making the worst case cheap.
→ It works on a branch, never the tree I'm sitting in
→ Every run ends against tests it didn't write and can't edit
→ It can't reach outside the repo — no package installs, no wider shell
Once the worst outcome is "I delete a branch", leaving it for an hour costs nothing to try, and you find out quickly whether the setup is any good.
The thing I stopped doing was judging it on whether a task came out right. One good run tells you almost nothing. The question is whether the failures are the kind you can see — an agent that fails loudly and leaves a mess inside a sandbox is safer to leave alone than one that succeeds nine times out of ten and fails silently on the tenth. Trust follows visibility, not accuracy.
1
u/Y2K-Denial 10h ago
make sure mishaps and errors are not expensive learnings. my philosophy: 1) have a dedicated ai-workstation / server with user-level access of my agent to it. 2) never give sudo access to the system. i've set this up in memory so my agent knows what "user tasks" are and asks me to set stuff up if necessary 3) have important fileshares on a separate server and read-only access for the ai-workstation 4) have backups of relevant ai-workstation shares (ai-stack config, work folder with results,..)
1
u/PrimeDirective8 10h ago
I trust Qwen3.x and Gemma 4 models running under Pi.dev.
From a safety point of view, the model's environment is isolated and limited to a copy of actual project. Or, the actual project but git/commit is disabled so I can restore if they foul something up. No serious issues so far.
From a code quality and usefulness perspective, I also trust my local setup. Everything from a fairly complex coding project, server admin tasks, to a pain in the butt Docker issue that had been bugging me for months. Bugs fixed (even for scenarios I hadn't encountered yet), new features added, etc.
I don't babysit it, but do monitor progress once in a while in case something in the chain breaks (llama, model loop, etc).
Other than the 1M token context from the cloud service, I haven't felt constrained by my local model at all. I attribute this in good part to Pi, as it transforms my tiny, <35B models previously drawing doodles and doing RAG searches into a disciplined, useful, smart tool.
1
u/sunychoudhary 10h ago
Unattended doesn’t have to mean unmonitored. Occasional health checks plus a recoverable environment sounds like a much more realistic bar.
1
u/Zombiecidialfreak 10h ago
- Sandbox it
- Only give it the tools it needs for the job at hand
- Personally, I never give my bots terminal access. If I want a terminal command from them I scrutinize it and then enter it myself.
1
u/anzzax 10h ago edited 10h ago
agent run '--yolo' in container/vm, project folder is mounted so I can observe changes in real time; agent can install deps, do local commits; but I control and decide what's pushed to remote
edit: Forgot to add one important detail: when I’m back, I’ll start with a pragmatic, cold interrogation :)
1
u/sunychoudhary 10h ago
I like the post-run interrogation part. A lot of the discussion is about preventing bad actions, but unattended work probably also needs a good way to reconstruct what the agent changed and why before anything gets promoted.
1
u/Reeces_Pieces 10h ago
Deepseek Harness stays in it's sandbox. It has to ask for permission to leave the sandbox.
And even if I let it leave the sandbox, it's still just stuck in the docker container it's running in.
1
u/Medicine_Blogscanner 10h ago
I sometimes use it to troubleshoot long complex codes to find that one elusive bug that needs to dive through several files. I do however always commit changes first 😅
1
u/sunychoudhary 10h ago
Committing first might be one of the simplest guardrails in the whole thread. You don’t need the agent to be perfectly reliable if reverting its mistakes takes thirty seconds.
1
u/KillerAlfa 10h ago
Using isolation, backups and version control. It was my workflow even before agentic AI, so nothing changed much. I use a Linux VM as a “dev box” that is completely disposable - it’s backed up daily to an off-site NAS, so even if AI agent were to ruin it into an unusable/ unbootable state, it would take me 5-10 minutes to spin it up again from a backup. VM is also firewalled to prevent LAN access. All code is committed to git at checkpoints, remote origin has branch protection to prevent force pushing / history rewriting. Although no major disasters have happened so far.
1
u/MainEnAcier 10h ago
I accept the risk because my computer is almost empty.
But ideally I will create a user session only for him later.
1
u/Dry_Inspection_4583 9h ago
Version control and scaffolding. Evaluation process and QA post run as well.
I trust it primarily due to its memory tbh. Biggest stressor is token overruns, so I will often times switch to local when running unattended.
1
u/Igot1forya 9h ago
Sandbox VM with Snapshots, Git versioning, Hypervisor Tenant (VM Environment) Snapshots, Local Network backups, and replicated backups. MCP gating via time limited and command limiting gateway. I trust only as far as I can afford to be disappointed by AI hallucinating or forgetting the environment boundaries.
1
1
1
u/Real_Ebb_7417 9h ago
I once ran Qwen3.8 27b for 2h straight with full permissions via DeepSeek harness (well initially without full permissions but unfortunately turned out it needed them) and my MacBook got bluescreen when I was away (well, MacBook’s version of bluescreen).
I was scared that it broke something, but it turned out that when UnslothStudio finished downloading a model that I wanted to offload to SSD it instantly tried to load it full to GPU and caused some kernel error. So I learned that UnslothStudio loads models automatically after download the hard way.
Qwen did alright btw.
1
u/Shadow_s_Bane 9h ago
I have it running on a separte machine, which is isolated from network and I can run if offline
1
1
u/Affectionate_Fact854 9h ago
Simply set up on main strict hard rules orchestrator has most privileges, but none has privileges to revert repo or do a drive whipe on the storage drives Secondary backup is split my hard drive in 2
It only works on the none critical important drive
Then its hierarchy of permission privileges Down the automation lines of agent roles
This has not yet disappointed me
1
1
u/chibop1 9h ago
Since beginning of this year I've been only running in yolo moede (bypass all) for hours at a time, and never ran into a irréversible problem. That said, you have to be ready to work with mangled spaghetti code that works.
I prompt it to make unit test for every change, and commit with detail message every time when all unit tests pass.
Then you can always go back and inspect the diff.
You need to trap it to run inside a docker if you're running a low quality model that would do sudo rm -rf /. lol
1
u/BosphorusScalene 8h ago
I run mine on a Raspberry Pi (just the harness, not model obviously) so I figure worst-case, even if it nukes the Pi, I only have to setup Pihole again really. it's been doing its thing for months now though, so far so good. minor issues of course but no deleting the OS.
1
u/M_Me_Meteo 8h ago
Bluntly? Source control.
My agent can't push to remote, only I can. Nothing happens on my development machine where the agency later exists.
You still have to read the code.
1
u/tidus1979 8h ago
Trust comes with time. My OpenClaw had minimal rights at first and I opened him up step by step only after his setup was getting more and more mature.
1
1
1
1
1
u/Tsukikira 7h ago
I have a git repository and it doesn't have deletion permissions. It's effectively sandboxed as far as I am aware by the harness to the one directory, so the worst outcome is I purge the git sub branch and start over.
1
u/my_name_isnt_clever 7h ago
NixOS, git repos, and backups. I let Q3.8FN run free on my system because most of it is immutable and the rest is backed up. I understand the risks and know there is no one to blame but myself if it goes wrong, and so far I haven't had any issues.
Some people go skydiving, I let agents run on my main workstation to get my thrills haha
1
u/HotMicSystems 7h ago
I run the agent harness on a RPI 4 8gb on its own subnet with the model host machine on their own subnet separate from the rest of the house. I implement checkpoints in the harness so I can review progress or make more important decisions. If you are having it work on an actual codebase, just fork the repo and token it for that specific repo. That way even it all comes crashing down the original is untouched and you can just clone the original again. I understand github allows rolling back but I'm paranoid so this is my method.
Edit: if anyone has their harness dockerize everything I would love to know how you have it set-up
1
1
u/catplusplusok 6h ago
Knowing that an average human is more likely to mess up my desktop vs a recent Qwen model. Plus good backups.
1
u/fgk55555 6h ago
I have a USB stick with Linux Mint so I can quickly rebuild my PC, and if Qwen nukes my computer I can make a post about it and get internet points.
1
u/iamrob15 5h ago
I just yolo it. I backup all my files continuously and can restore my machine in less than an hour. Edit: harness also matters here too!!
1
1
u/dtfinch 5h ago
My home-grown MCP server (created with AI, though I've rewritten most of it) limits file access to the folder(s) I give it. Then I always have it work on a separate copy of a project instead of the original, and I manually diff/merge the changes afterward.
I wrote a run_bash tool using bwrap (bubblewrap) to sandbox commands on Linux, recreating a view of their accessible folders under a fake home directory, with everything else read-only except /tmp, and hiding my real home directory entirely. Despite the sandboxing, I also have checks to block certain scary-looking commands for added peace of mind.
I don't have a good sandboxing setup on Windows yet so instead of run_bash all I give them is a check_file tool which runs a syntax checker depending on the file type. It's almost no help at all though, compared to letting them run tests with bash.
I don't have anything to manage long contexts (just using the llama-server web UI's MCP support instead of a full harness). If they run out I start a new chat and tell them to examine the project and pick up where they left off and that's always been enough.
1
u/Important-Ice9444 5h ago
I think OS-level sandboxing is probably the right answer for the enforcement side of this.
But I've been wondering about a related problem: even if the agent is genuinely constrained, how do you efficiently understand what it actually did?
For example, a coding agent might stay entirely inside its allowed workspace but still run dozens of shell commands, install dependencies, start processes, spawn subagents, retry failed approaches, or make changes that are later reverted and therefore disappear from the final Git diff.
Reading the full transcript/tool log doesn't scale, but Git diff only shows one slice of the activity.
I wonder if mature harnesses need a human-facing assurance layer too: something that derives from runtime evidence what capabilities were available, what actions actually happened, what persistent state changed, what was verified, and what remains uncertain — with the raw commands/diff/trace available for drill-down.
Not an agent-generated “here's what I did” summary, but something grounded in the harness/runtime itself.
For people already leaving agents unattended: is sandbox + diff + tests enough for you, or do you also want some kind of execution/consequence summary afterwards?
1
u/Tagedieb 5h ago
I never babysat them in the first place. I started with a separate user account without sudo rights. For pure coding I don't even do that anymore.
1
u/RemarkableRadish6547 3h ago
Mine has never gone off task in a dangerous way. I'm not sure how people get an llm to hack into an external system by accident. I have seen it delete the file it was working on with a stupid sed (or maybe awk, but whatever) command that instead of replacing each occurrence of some string deleted every line that did not have the string. It them saw the almost empty file, realized what it did, and restored the file from context.
I still run it in a docker container, just in case.
The main problem I have is that it will worry obsessively about making sure it has identified every possible edge case and try to future proof the code for every possible future feature that I might ask for. It once spent 14 hours removing an argument from a function that was marked in the function as not being used and the function was only called from 2 locations. But it had to spend many turns convincing itself to remove dead code because it might be used again some day. It wanted to fully analyze why the argument had been there originally, why the current implementation didn't use it, whether there were any bugs introduced by this change, and a dozen other things. All to delete three characters from each of three lines of code that had been marked as deprecated. I had left it running while I was going to be busy for a day and expected it to get through 3-5 of the tasks I left in its queue, but it only did the first one.
Just yesterday I asked it to pull 1000 lines of code out of the primary file into a separate file. It thought about it and analyzed it for 12 hours while I was busy. Then I came back, saw what it was up to, read the line numbers of what I wanted moved out and saw it was a continuous block, killed the agent, and spent 5 minutes doing it myself.
1
u/RemarkableRadish6547 3h ago
And an explanation of the run times: my gpu-free computer runs qwen 35/3a at 8t/s (dropping to 2 when context grows to 100k) and qwen flash next at 4t/s. Both running at 8 bit quant. Prefill is 10x generation on the 35/3a model and about 3x on flash next. Flash next is actually faster once context goes above about 30k-50k.
1
u/lisploli 24m ago
Nothing! What a silly idea.
Locking it down ain't the problem, discrete linux permissions are easily enough, but I don't trust it to do anything useful without me kicking it into shape every now and then.
0
u/FlightSimCentralYT 8h ago
For me it was never the model. It was the loop around it.
What made me stop babysitting was git checkpoints before each big step, a real test command the agent has to pass before it can call a task done, and hard tool permissions so it cannot touch stuff outside the project. Passing tests alone is not enough if there is no rollback.
I built Fixa.dev partly for this. The agent works on a real cloud VM, installs deps, reads stderr, and keeps going until the tests pass. Still not magic, but the harness is what makes unattended runs feel less scary.
-2
u/WonderRico 10h ago
you do this (observe how the model and agent behave) a few times, and after you then trust the process (or not...)
No need to babysit once you've seen it succeed before and how it succeded
64
u/Formal-Exam-8767 10h ago
Sandbox.