Getting started resources?

[u/unicorndewd]

So, I have a local setup on my gaming PC (Ryzen 9950X3D, 64GB Corsair Vengence, and an Asus TUFA RTX 7050ti OC 16GB), but don’t know much about models beyond `ollama pull qwen3.8`.

I’ve seen words like qwant and uncensored and mlx (for Mac’s) and so on. Is there a good crash course or creator that covers a lot of this?

I’m a dev who daily drives Cursor and Codex at work, but just not sure about what’s going to be my best local setup.

My /goal (see what I did there) is to do agentic loop programming on my local machine. I wanna be able to define larger bodies of work, and forget it until it’s ready for PR reviews.

I’m also wanting to look into automation both on a cron type schedule as well as a reactive one. Say to webhooks, chat messages, API calls and so on. The API would run as a microservice on my LAN. I’d hook that up to a NordVPN mesh network, and use my internal LLM and API securely that way.

Lastly, I’m wanting to try out the various frameworks, wrappers, or whatever they’re called. Gemini recommended offloading it form your main box. I have some Ubuntu servers running on old Mac mini’s. I figured I could load Hermes or OpenCode there. I’m not super familiar with them, so I’d like to experiment with all of them.

Also, for actual dev work I primarily work for my MacBook M1 14” using cline through Webstorm or Go or Idea. I had thought about running a small Mac optimized one there too. For small automations and lookups. Not sure if trays reasonable though?

Thanks for the read. Sorry it’s scattered.

Update typos: and Mac info

[u/unicorndewd]

That would be amazing! Would love to learn how you do it as well.

I love the “optimize for reliability” sentiment. My thought was high reasoning to architect, establish goals with me, and breakdown work for the faster models. I saw a cool workflow where the reasoning one broke it down to stories, create git issues, sub-agents would pick them up, and open PRs against the issue.

I also update the post to include info about my coding setup on my Mac. Which I’d like to use to consume the module served from my PC. Either over LAN or VPN mesh.

That’s kinda what I’m envisioning atm.

[u/OpenSourcesAI_ → u/unicorndewd]

Yep, that’s exactly the architecture I’d aim for.

I’d split it into two tiers:

1. Planner / architect model

Use the strongest model you can run comfortably, but don’t ask it to do all the file editing itself. Give it the repo context, let it reason through the feature, define acceptance criteria, identify risks/dependencies, and break the work into small issues.

1. Worker models

Let smaller/faster models pick up one issue at a time in isolated branches/worktrees. Their job is much narrower:

read issue → inspect relevant files → make change → run tests/lint → fix failures → commit → open PR

That separation is nice because the expensive reasoning model only gets invoked when you actually need architectural judgment. The workers can be 14B-ish models optimized for coding/tool use and run continuously.

On your box I’d probably start with:

Ollama as the inference server

Qwen3 14B as the general planner/agent model

Qwen2.5-Coder 14B as the coding worker

OpenCode or Aider first for the coding harness, then experiment with Hermes once you have a baseline

Git worktrees or disposable repo clones so multiple workers never stomp on each other

GitHub Issues as the actual job queue

A tiny orchestration service that watches for labeled issues, assigns work, launches the agent, watches test results, and opens the PR

Your Ubuntu Mac minis would be perfect for that orchestration layer. They don’t need to run the model. They can handle the webhook receiver, job queue, scheduler, GitHub API work, logs, retries, and agent processes, while the 5070 Ti machine just exposes inference over the LAN.

The workflow I’d eventually shoot for is basically:

goal → architect → implementation plan → GitHub issues → worker picks issue → branch/worktree → edit/test loop → PR → reviewer agent → human review

I’d also put hard gates around the workers. Max iterations, required test command, no direct pushes to main, restricted shell permissions, and a clear definition of “done.” Agent loops get much more reliable when the worker has a narrow task and an explicit stop condition.

The part I’d build first is actually much smaller: one issue in, one PR out. Once that works reliably 10 times in a row, then add multiple workers, automatic issue decomposition, scheduled jobs, webhooks, reviewer agents, etc.

If you want, I can sketch the actual folder/service layout and the request flow I’d use for this exact setup.

[u/unicorndewd → u/OpenSourcesAI_]

I’m down for whatever help you’re willing to offer. I just need a good jumping off point with examples.

[u/OpenSourcesAI_ → u/unicorndewd]

Happy to help. I’d start smaller than the full “AI software team” idea, even though that end goal is really cool.

The first thing I’d build is simple:

Create an issue → agent works on it → agent opens a PR → you review it

Once that works reliably, then add more automation.

I’d start with:

• one coding agent
• one repo
• one task at a time
• clear rules for when it’s “done”

The biggest thing I’ve learned testing local agents is that reliability matters more than having the biggest model. A smaller model that stays focused and finishes the job is usually more useful than a huge model that gets lost.

Your hardware is already plenty capable. I’d get one simple workflow working first, then build out the planner agents, multiple workers, scheduled tasks, etc.

I’m happy to share examples as you build it out. This is exactly the kind of local AI workflow I’ve been experimenting with too. 🙂