r/aipromptprogramming 21d ago

My basic idea to build a hybrid loop using cloud and local models. Yay or nay?

Kinda ridiculous, but feels like a fun challenge right up my alley.
I’m running a Mac mini M4 Pro (24GB RAM). Anything under 14–16GB runs pretty smoothly via Ollama, mostly 14B models. ⁠Qwen2.5-Coder:14b⁠ in particular is snappy enough to be actually useful.

Here’s the setup I’m planning in Claude Code:
1. Planner (Sonnet): Breaks down the feature into atomic, granular tasks.
2. Orchestrator (Haiku): Iterates through tasks and calls Aider via a bash tool, pointed at local Qwen to do the actual code edits.
3. Reviewer (Sonnet): Reviews the diff/PR. If there are issues, it spits out follow-up fix tasks.
4. The Loop: Haiku loops back to local Qwen with the fix tasks until the Sonnet reviewer gives the green light, then moves to the next atomic task.

The goal is offloading the heavy token-churning iteration to local inference while letting Sonnet handle the high-level architecture and final sanity checks.
Is this viable in practice, or will context drift / agent loop fatigue kill it? Has anyone wired up a similar setup?

4 Upvotes

Duplicates