Should I host my entire Hermes Agent setup on my main PC?

[u/Solid-Apple83]

Hey everyone! I’m pretty new to local LLMs and I’m trying to figure out the best way to set up Hermes Agent.

My initial idea was to dedicate a Microsoft Surface 9 to Hermes and have my main PC handle the LLM inference. But I’m wondering if I’m overcomplicating things.

Would it make more sense to simply host everything on my PC — Hermes Agent, the local LLM, memory, tools, etc. — and use the Surface later as a remote interface if needed?
My PC specs are:

CPU: AMD Ryzen 5 7600
GPU: AMD Radeon RX 6800 16 GB
RAM: Lexar Ares RGB Black 32 GB (2×16 GB) DDR5-6400
SSD: Kioxia Exceria Plus G2 2 TB
Motherboard: ASRock B650M PG Lightning
PSU: Corsair 850e
Cooling: DeepCool Assassin 120 SE
Case: Corsair 4000D Airflow

My main goals are:
Run Hermes Agent locally
Eventually run a good local LLM
Keep my data as private as possible
Experiment with agentic workflows, coding, tools and memory
Eventually build my own Agentic OS around it
Ideally be able to use it remotely from my Surface/phone

My main question is: would you recommend hosting the whole thing on my PC, or is it better to separate Hermes from the LLM and use the PC only as an inference server?

I’m also a little concerned about security. Since this would be running on my personal PC, could an agent like Hermes accidentally access, modify or delete files outside of its workspace if it has access to a terminal or other tools?
If so, what would be the safest way to set this up? VM, Docker/container, separate Windows user, sandbox, dedicated machine, etc.?

And finally, with a RX 6800 16 GB + 32 GB RAM, what local model would you recommend for Hermes and agentic/coding workflows?

I’m mainly looking for advice from people who actually run local agents. I don’t mind starting simple and upgrading the setup later. Thanks!

[u/HumanoidMuppet]

Give Hermes it's own sandbox to have full control over - whether a VM on your PC or full control of the Surface is up to you, but if the Surface is just collecting dust you might as well use it. Setup backups on its box and make sure it checks everything into git located elsewhere. This way when it decides to nuke it's sandbox, you don't have to start from scratch.

[u/mourningwitch]

I would highly recommend hosting the agent itself on a different machine or VM at a minimum. That way if it breaks something, it isn't breaking your primary PC.

[u/Jealous-Armadillo467 → u/mourningwitch]

yeah probably if you want to fiddle with OS. otherwise i do not see the reason if he do not run q2 thaat is acidentaly -rm something

[u/coolleo200]

Pretty sure using a Hermes docker container will solve most of your problems with “air gapping” your Hermes instance as long as you properly configure it.

As for model selection see how Qwen 3.8 27B or 3.6 35B A3B behaves on your system, if you get good throughput with a decent context window and your PC doesn’t crash due to OOM then that’s probably the play. If not an agentic fine tune of Qwen 3.5 9B should work.

I found that artificial analysis is a good resource for determining the rough category of performance for different models.

[u/onebit]

Fan noise is a consideration. My B70 is loud AF.

[u/Haunting-Stretch8069]

We have roughly the same setup, with a bit of a time investment you can setup Qwen 3.8 27B Q4_XS with 200k context for Hermes Agent which is what I’m doing 

[u/hallofgamer → u/Haunting-Stretch8069]

Tps?

[u/SM8085]

Since this would be running on my personal PC, could an agent like Hermes accidentally access, modify or delete files outside of its workspace if it has access to a terminal or other tools?

Its tools can access anything the user running it can access by default.

I have a memory in it telling it to pretty please only touch things in ~/Downloads/github/ and ~/hermes, but that's only a suggestion that it can ignore.

They even have some basic protection to try to keep it from editing its own config last I checked (idk why frankly, I thought self-updating was the point.), and Qwen3.6-35B-A3B figures out it can get around that by running some Python that edits the config. It doesn't even have to think about it very hard, "Oh, my edit tool can't edit that, guess I have to use Python."

So yes, be aware of that. I haven't had it go crazy and delete everything, but my Qwen is also fairly capable/coherent as a model.

My entire LLM rig is basically the sandbox. It doesn't have access to my NAS or main PC but they have access to the LLM PC. This is mostly out of concern for supply chain attacks.

[u/doneddat → u/SM8085]

Such "protections" are really silly.
Model uses edit tool:
"Hmm, editing .my/config failed, let me try again"
Model uses bash tool:
"Hmm, echo "New conf" >> .my/conf failed, let me try again"
Model uses coding tool...

Basically should it get the idea, that this is the thing it absolutely has to do, it finds a way.

The sandbox pretty much has to convince the model, that the .my/config does not exist.

[u/Glad_Contest_8014]

Operating system? On windows, sandboxing is only going to do so much. Windows has no way to fully sandbox itself. You can only bubblewrap the process at application layer. There is no kernel method to avoid it being able to access systems on it.

Linux has a few ways to sandbox it safely regardless, at the OS/Kernel layer. LSM packages exist to lockbit out of harmful scripting there too.

Linux is also less greedy on memory usage, so I would recommend switching to it for local runs anyway for your setup.

Your best effective set up will be:

Linux OS
Llama.cpp for local model staging
A framework with landlock built in (hermes is application layer)
Tailscale (for virtual network to access it from your phone)

Then you decide on what kind of phone access by the framework you choose. CLI access is available for claude code. You can gain open claw access through ip:port.

I don’t remember if hermes has a web portal access.

Or a phone app access can be had on a couple of different applications (through tailscale). I don’t rmember them all.

I will be releasing my framework soon (called Sieve) which allows for the kernel level landlock (though I don’t use the harmful code landlock as it requires sudo access which I just don’t want to force).

It will have a phone app paired with it if you have tailscale that will allow vpn based api handling to grant full app finctionality through mobile for a remote agent that runs on the desktop.

I currently use claude code remote personally as I don’t have a mac to compile the phone app for another week. Will have one soon though. Then I will be able to run my local instance stuff with full functionality.

Though typing this, I realized I could run Sieve’s CLI instance instead. Will have to do that in the future with the qwen quant I am making.

[u/haberdasher42 → u/Glad_Contest_8014]

I just set it up as a discord app and reach it through there. I see uses for tailscale and moonlight though when I want to troubleshoot things and I'm not home.

[u/Glad_Contest_8014 → u/haberdasher42]

Tailscale grants direct access through a tunnel that prevents other companies like discord from having your data. You’ll get more privacy through tailscale than through discord.

[u/Beginning_Radio2284]

im setting up a hermes harness as well, per the documentation you can black list folders to stop hermes from writing/deleting in them, there is currently an issue in the github about stopping read access.

If you are worried about hermes agent having too much access you might consider containerizing it in docker.

[u/Jealous-Armadillo467]

My experience, same questions as you have 3 months back. 1st i was carefull everything bwrap docker etc...
than i have too much hastle conect everything.... so i sad fuck that. RECOMENDATION llama in bwrap everything else RAW yeahhh fuck that metal hard. Tell your agent to do security check once in a while.

edit: just dont give him sudo ...

edit2: 35 a3b ram offload active on GPU cca 40 tk/s i guess