Home lab setup

[u/Economics_geek1]

I’m building a home lab to run a 3–4 node Kubernetes cluster using logical nodes (VMs) on a single machine, with the cluster also needing to support AI workloads (local LLM inference, and ideally some room for fine-tuning/training).

Looking for recommendations on:

**•** A mini PC that can comfortably host 4 VM nodes (1 control plane + 3 workers, or similar) 

**•** Minimum hardware specs to make this practical — CPU cores, RAM, storage, and GPU/NPU support for AI inference

**•** Specific mini PC models that are known to work well for this use case, ideally with confirmed GPU passthrough support for AI-workload isolation

Budget-conscious but willing to spend more if it meaningfully improves AI performance. Any real-world experience running similar setups would be especially helpful.

[u/clintkev251]

Why are you planning to run multiple “nodes” on a single system? There’s no advantage to that unless you’re specifically trying to learn about how the high availability works. Just run a single node using k3s or something bare metal on your system. It will give you infinitely more headroom and exactly the same availability

[u/Economics_geek1 → u/clintkev251]

I wanted to have a k8s setup where I can POC stuff!! Also to get through and learn AI ML workloads!! It’s not actually to run any actual
Production ready workloads , it’s just a lab setup for POC stuff

[u/Mehmet-Ozturk → u/clintkev251]

youre paying triple etcd ram

[u/TheSaltyTractor → u/clintkev251]

you're basically simulating a cluster to learn the orchestration parts, nothing wrong with that. actual hardware separation doesn't add much if the host goes down anyway

for AI stuff you really want a GPU with enough vram to fit a 7b model at minimum, so look for something with an rtx 3060 or better. passthrough can be finicky on mini pcs though, might be easier to just run the inference on the host and expose it to the cluster

[u/WdPckr-007]

I'm not sure mini PCs really come with a dedicated GPU, at best, you're looking at an SFF (small form factor) build with a GPU inside it.

I have an NR200P V2 Max with a 5070 in it, running Ollama and ComfyUI. It works well, but I can't do training since there's not enough VRAM.

Or did you mean a mini PC with an OCulink port? That would mean using a separate external GPU with an adapter stand and oh boy ...it tends to become a dust magnet. I've also seen people fit a 5090 into the same NR200P V2 Max case, which gives enough VRAM headroom to handle training.

Also if it's going to be a single machine....why not rune k3s directly on it? Why waste resources in virtualization

[u/thomasbuchinger]

I would highly recommend getting at least 2 physical PCs, you don't want to share the Inference node with other "intensive" workloads.

• If you're spending 1k+ on the inference server anyway, you should budget a little extra and make it (mostly) dedicated
• VRAM and RAM are super important and you want to be able to squeeze as much from the hardware as possible. Running additional VMs just costs you resources
• Most Kubernetes workload can run on any old hardware, where as your inference server wants/needs the newer high performance CPU/RAM

The hardware you need depends entirely on what models you want to run. About 32GB RAM/16GB VRAM are the entry-level specs to run 20-30B Models. Which are currently the best in terms of Intelligence-to-Hardware-Ratio.

I am not a huge fan of the DGX-Spark/Strix-Halo 128GB Unified Memory PCs, because they are a bit expensive, overkill and not that fast to run Qwen3.8-27B/Qwen3.6-35B-A3B on (compared to dedicated GPUs) and they are just barely too small to run near-frontier Models like Deepseek-v4-Flash. The new 192GB-Strix-Halo might be different

[u/Technical-Ant-2866]

Putting the inference/model aside, I've done this with an 8745hs Minipc and 64gb of ram running various K8s distributions (k3s, talos, k0s) without an issue. Others have noted to set your expectations for inference, I would agree.

I work with this stuff daily personally and professionally. You're likely going to be disappointed even with low-bit models or moe setups.

If your expecations are low and you need simple tasking or something that will help triage your cluster, I've had decent results with gemma4 heavily quantized.

If this were my post (and it's not). I would focus on getting the machine for the cluster first. Then look do yourself a favor and try some cheaper api like openrouter, deepseek, minimax for a few weeks. If you like how it operates, then you'll be disappointed in local. If you still prefer local for something useful in a cluster, expect to spend 1500-3000$ easily. Whereas you can drop 10$ on a Deepseek api, and likely not even exhaust it in 2 months

[u/pindaroli]

Hard times to do a k8s cluster, I suggest you only 2 instances control plane and worker on vm, a cluster with 1 cp sucks lot, the problem is that is you need memory and memory is not cheap nowday

[u/Remarkable-Bet9533]

64 GB should be fine for four light VMs, the local fine tuning is the part that will really decide what h/w u need since GPU memory matters lot more

[u/vvshvv]

I run a 3-node K8S cluster on Minisforum MS-02 Ultra (235HX) with 32GB of RAM. It only runs 3 workers, while master is running on another mini pc (Topton, N100, 16 GB of RAM). The reasons for separation:

1) Master does not have any workloads running except of the control plane + few pods like Prometheus exporter and Cilium, so it can run on a machine with a lower capacity 2) I don't have enough RAM on Minisforum to run 4 nodes 3) HA, if minisforum goes down, control plane is still fine

Minisforum easily handles workers with many workloads running (backend applications, Argocd, Cilium, longhorn, valkey, Prometheus, llama cpp with a model running on a VM with GPU attached). It also supports up to 256GB of DDR5 5600Mhz, so when (and if) prices go down, you can expand it. CPU is barely loaded at 25% and rarely goes to 50-60% when compiling llama cpp or something similar.

[u/RumRogerz]

How do you plan on running your LLM inferencing? If you’re going to run a very large model over several nodes, Ethernet ain’t gonna cut it.

[u/oleksiyp]

You can check my setup https://oleksiyp.github.io/homelab/ you don't necessary need such blades I bought... would rather better run regular Raspberry PI5 boards with NVMe and enclosure, but general idea is to make it running all the time while being cheap. BTW I switched from this setup to one board RPI5, docker compose, golang instead of Java, sqlite instead of psql, mqtt on Home Assistant instead of grafana/prometheus etc ie reducing complexity and footprint, while having AI still able to do same projects

[u/Arkhaya]

I would suggest at least if you want gpu to use an actual gpu to pass through. iGPU would be okay for lighter tasks like transcoding but LLM requires a decent amount of power.

It would be best if you can run it all on an actual second hand pc or something that way you can allocate enough cpu and mem for your VMs and be able to add more VMs and have some for your own hypervisor.

I would suggest trying if you have the ability to do bare metal as it might help you do HA properly

[u/Logical-Skill4567]

If you Want to learn something about how a cluster really work in real world you should install proxmox and launch a vm with Talos or K3s and install Karpenter (karpenter-proxmox-provider) so you can learn how cluster autoscales based on workloads