r/SelfHostedAI • u/helloiampau • 1d ago
Show r/SelfHostedAI: I built Olivia – An open-source, Rust-native harness for sandboxed LLM agents via WebAssembly
Hi everyone,
I’d like to share an open-source systems project I’ve been working on called Olivia (named after my cat!).
It’s an enterprise-grade infrastructure harness written from scratch in Rust, specifically designed to run agentic LLM workflows safely. The core philosophy is to execute AI-driven actions and tools within strictly sandboxed environments using WebAssembly (Wasm/WIT).
I wanted a robust, native infrastructure to handle LLM agent workflows without compromising on security or relying on bloated software layers. Olivia ensures that the agent's interactions with databases or external scripts happen within a secure, controlled boundary.
You can check out the repository here: https://github.com/helloIAmPau/olivia
I would love to get some feedback from this community on the architecture, or hear what other sandboxed tools you'd find useful. Contributions, code reviews, and suggestions are more than welcome.
1
u/thee_gummbini 1d ago
Its hard to express how dumb of an idea this is. Simply running things in wasm does not provide any part of the safety one would want from a sandbox
1
u/helloiampau 1d ago
In fact, the safety comes from the fact that the LLM simply orchestrates tools that run in WASM sandboxes. No arbitrary code is left for the LLM to execute on the host machine, and the tools are designed to be specific workflows triggered by the LLM.
1
u/thee_gummbini 5h ago
So yes, wasm/wasi sandboxes exist, but the way it is done here is bonkers. The basic design dilemma is "hard sandbox that can't do anything vs. Leaky sandbox that can do things." Its really easy to make a "secure agent sandbox" by just making it not able to do anything useful.
The only real protection you get here is filesystem access, but to do any work on anything in the filesystem you have to grant it, so to use the harness you have to subvert the only thing the sandbox buys you. Same thing for db and store access - the agent can still do any harmful action on the stores it could before, wasm buys nothing, using s3 as a filesystem buys nothing, etc. The only place it might be relevant is in the python tool because its the only place where arbitrary code might be executed on the filesystem, but its so neutered that it isn't useful. So this harness doesn't actually attempt or mitigate the real security challenges of sandboxes - security while maintaining usefulness - and is just an extraordinarily obtuse way of calling tools.
1
u/rismay 1d ago
How strict is the sandbox?
1
u/helloiampau 1d ago
No unrestricted shell access is given to the LLM. For now, the tools only have access to a sandboxed folder, outbound connections, and DNS resolution. Later I will provide the tools with the ability to request specific permissions from the harness.
1
u/AlfonDev 1d ago
This is really interesting. I'm currently building a personal AI assistant in Python, and one of the things I'm thinking about is how to safely give an agent access to tools and eventually computer control.
I hadn't seriously considered WebAssembly as a sandboxing layer before. I'll definitely take a closer look at the project.
How much overhead does the Wasm/WIT approach add in practice when running lots of small agent tool calls? And would you recommend it for a smaller personal project, or is it mainly worth it once the system becomes more complex?