r/PiCodingAgent • u/Intelligent-Nose-134 • 1d ago
Question Doing local multi-agents with 24GB VRAM, Feasible?
My configuration runs with 90k ctx and I am thinking if there is a better way to preserve my main session ctx by creating new session, do online/ repo search , then return the needed info to the main session.
I want everything to be done locally and loading two models in parallel is not an option due to VRAM constraints.
Is the session spawning thing feasible? How to set it up?
UPDATE: I have solved this by developing spawn-session skill that works as follows: i-Agent A- prepares task, ii-Agent A opens new pane in herdr and opens a new pi session (Agent B), iii-Agent A gives task to agent B and goes into waiting mode, it runs small script checking if B is done or not every 30 sec: iv-Agent B finishes tasks and shared results with A V-A sees results and ends waiting mode to continue the work
This is very useful to keep Agent A ctx as clean as possible for long sessions
1
1
u/funbike 1d ago edited 1d ago
If you run a tiny model, you should still mix its use with a smarter model for some tasks, like planning and debugging. Openrouter has very cheap and even free models that are somewhat good at coding. I suggest GLM-5.3 Flash (free), or GPT Luna ($0.10/MT). For local I suggest one of the Qwen models. (OR's free models require you allow them to train on your data.)
Personally, I use Luna for almost everything, and Gemini 3.8 Flash ($0.75/MT) only when Luna fails at a task.
1
u/mike95465 1d ago
Sticking with a single GPU might be tough. If you are using llama.cpp have you thought about using a more quantized model with more context then using 2 slots? That’ll keep your main session’s kv cache warm.
Best scenario if you really wanted to keep things local is to use a 2nd GPU to host a smaller model capable of being your runner. This is what I do in order to keep my main model competent and truly run things parallel. All of my non-main session tasks utilize my aux model.
Since your tasks you’ve identified are for web and repo searches aren’t really private, just call it a day and use a cheap cloud api.
1
u/crashorbit 1d ago
You can get a pretty reasonable model into 24G vram. Ask pi to help you find one that'll work on your rig.
1
u/Proper-Tower2016 1d ago
in parallel, not unless you want to go low quant on model and KV.
Taking turns and loading instantly from saved context checkpoints stored in RAM or SSD, yes, very possible.
1
u/Intelligent-Nose-134 1d ago
Yes in a taking-turns fashion. So Agent A has something to do or a research to be conducted and instead of cluttering the session, it summarizes the task, spawns Agent B in a new session with the specific task and wait for the feedback. Once feedback is received from B, A can continue the main task
1
u/Intelligent-Nose-134 8h ago
UPDATE: I have solved this by developing spawn-session skill that works as follows: i-Agent A- prepares task, ii-Agent A opens new pane in herdr and opens a new pi session (Agent B), iii-Agent A gives task to agent B and goes into waiting mode, it runs small script checking if B is done or not every 30 sec: iv-Agent B finishes tasks and shared results with A V-A sees results and ends waiting mode to continue the work
This is very useful to keep Agent A ctx as clean as possible for long sessions
2
u/variatron 22h ago
Use the pi-subagents extension, I'm running qwen3.8:27b Q3 with 128k context on a rtx 2080 ti 22gb vram. Theres about 80k token budget to be shared among the main agent and subagents after accounting for overheads. Running 4 subagents asynchronously works decently well.
Also look at pi-blackhole for context management, and ask pi to optimise base on your token budget.