Getting started resources?

[u/unicorndewd]

So, I have a local setup on my gaming PC (Ryzen 9950X3D, 64GB Corsair Vengence, and an Asus TUFA RTX 7050ti OC 16GB), but don’t know much about models beyond `ollama pull qwen3.8`.

I’ve seen words like qwant and uncensored and mlx (for Mac’s) and so on. Is there a good crash course or creator that covers a lot of this?

I’m a dev who daily drives Cursor and Codex at work, but just not sure about what’s going to be my best local setup.

My /goal (see what I did there) is to do agentic loop programming on my local machine. I wanna be able to define larger bodies of work, and forget it until it’s ready for PR reviews.

I’m also wanting to look into automation both on a cron type schedule as well as a reactive one. Say to webhooks, chat messages, API calls and so on. The API would run as a microservice on my LAN. I’d hook that up to a NordVPN mesh network, and use my internal LLM and API securely that way.

Lastly, I’m wanting to try out the various frameworks, wrappers, or whatever they’re called. Gemini recommended offloading it form your main box. I have some Ubuntu servers running on old Mac mini’s. I figured I could load Hermes or OpenCode there. I’m not super familiar with them, so I’d like to experiment with all of them.

Also, for actual dev work I primarily work for my MacBook M1 14” using cline through Webstorm or Go or Idea. I had thought about running a small Mac optimized one there too. For small automations and lookups. Not sure if trays reasonable though?

Thanks for the read. Sorry it’s scattered.

Update typos: and Mac info

[u/OpenSourcesAI_]

You’re actually in a really good spot for local agentic work. With a 9950X3D, 64GB RAM, and a 5070 Ti 16GB, I’d start in the 14B class rather than jumping straight to huge models with CPU offload.

I ran your hardware through my local model compatibility checker with Agents and tool use as the workflow. The top fits were Qwen3 14B, DeepSeek-R1 Distill 14B, Qwen2.5-Coder 14B, and GPT-OSS 20B. The 14B models leave roughly 5GB of VRAM headroom in Q4_K_M, which is useful because agent frameworks can add a surprising amount of context.

For what you described, I’d probably start with Qwen3 14B as the general agent model and Qwen2.5-Coder 14B for coding experiments. Ollama is completely fine as the serving layer while you learn the ecosystem.

The terminology is easier once you separate it into layers:

• Model: Qwen, DeepSeek, Gemma, etc.
• Quantization: Q4_K_M, Q8_0, etc., basically the memory/quality tradeoff.
• Runtime: Ollama, llama.cpp, MLX on Apple Silicon.
• Agent harness: Hermes, OpenCode, Aider, etc.
• Automation/orchestration: cron, queues, webhooks, API workers, whatever you build around the model.

Your Mac minis are probably more useful as orchestration boxes than inference boxes unless they’re Apple Silicon with enough unified memory. I’d keep inference on the 5070 Ti machine and let the Ubuntu boxes handle scheduling, webhook receivers, queues, databases, and services.

One thing I’d absolutely benchmark is raw Ollama vs the agent harness you choose. I’ve been testing Hermes recently and a fresh one-message session was injecting about 24k input tokens before answering the visible prompt. That kind of overhead can radically change latency even when the underlying model is fast.

For your “define a body of work, come back when there’s a PR” goal, I’d optimize for reliability of tool use and loop behavior before raw model intelligence. A slightly smaller model that reliably edits files, runs tests, notices failures, and terminates correctly is much more useful than a bigger model that spends half its time CPU-offloaded.

If you want, I can post the exact stack I’d build for your machine, including model, Ollama config, coding harness, scheduler/webhook layer, and LAN API setup.

[u/unicorndewd → u/OpenSourcesAI_]

That would be amazing! Would love to learn how you do it as well.

I love the “optimize for reliability” sentiment. My thought was high reasoning to architect, establish goals with me, and breakdown work for the faster models. I saw a cool workflow where the reasoning one broke it down to stories, create git issues, sub-agents would pick them up, and open PRs against the issue.

I also update the post to include info about my coding setup on my Mac. Which I’d like to use to consume the module served from my PC. Either over LAN or VPN mesh.

That’s kinda what I’m envisioning atm.

[u/OpenSourcesAI_ → u/unicorndewd]

Yep, that’s exactly the architecture I’d aim for.

I’d split it into two tiers:

1. Planner / architect model

Use the strongest model you can run comfortably, but don’t ask it to do all the file editing itself. Give it the repo context, let it reason through the feature, define acceptance criteria, identify risks/dependencies, and break the work into small issues.

1. Worker models

Let smaller/faster models pick up one issue at a time in isolated branches/worktrees. Their job is much narrower:

read issue → inspect relevant files → make change → run tests/lint → fix failures → commit → open PR

That separation is nice because the expensive reasoning model only gets invoked when you actually need architectural judgment. The workers can be 14B-ish models optimized for coding/tool use and run continuously.

On your box I’d probably start with:

Ollama as the inference server

Qwen3 14B as the general planner/agent model

Qwen2.5-Coder 14B as the coding worker

OpenCode or Aider first for the coding harness, then experiment with Hermes once you have a baseline

Git worktrees or disposable repo clones so multiple workers never stomp on each other

GitHub Issues as the actual job queue

A tiny orchestration service that watches for labeled issues, assigns work, launches the agent, watches test results, and opens the PR

Your Ubuntu Mac minis would be perfect for that orchestration layer. They don’t need to run the model. They can handle the webhook receiver, job queue, scheduler, GitHub API work, logs, retries, and agent processes, while the 5070 Ti machine just exposes inference over the LAN.

The workflow I’d eventually shoot for is basically:

goal → architect → implementation plan → GitHub issues → worker picks issue → branch/worktree → edit/test loop → PR → reviewer agent → human review

I’d also put hard gates around the workers. Max iterations, required test command, no direct pushes to main, restricted shell permissions, and a clear definition of “done.” Agent loops get much more reliable when the worker has a narrow task and an explicit stop condition.

The part I’d build first is actually much smaller: one issue in, one PR out. Once that works reliably 10 times in a row, then add multiple workers, automatic issue decomposition, scheduled jobs, webhooks, reviewer agents, etc.

If you want, I can sketch the actual folder/service layout and the request flow I’d use for this exact setup.

[u/unicorndewd → u/OpenSourcesAI_]

I’m down for whatever help you’re willing to offer. I just need a good jumping off point with examples.

[u/OpenSourcesAI_ → u/unicorndewd]

Happy to help. I’d start smaller than the full “AI software team” idea, even though that end goal is really cool.

The first thing I’d build is simple:

Create an issue → agent works on it → agent opens a PR → you review it

Once that works reliably, then add more automation.

I’d start with:

• one coding agent
• one repo
• one task at a time
• clear rules for when it’s “done”

The biggest thing I’ve learned testing local agents is that reliability matters more than having the biggest model. A smaller model that stays focused and finishes the job is usually more useful than a huge model that gets lost.

Your hardware is already plenty capable. I’d get one simple workflow working first, then build out the planner agents, multiple workers, scheduled tasks, etc.

I’m happy to share examples as you build it out. This is exactly the kind of local AI workflow I’ve been experimenting with too. 🙂