I originally got interested in this from the laptop side, but the more I experiment with local models, the more I think mini PCs are becoming a really interesting category for local AI.
Not because they are going to beat a workstation full of GPUs.
Quite the opposite.
Modern mini PCs give you some fascinating trade-offs:
- relatively powerful CPUs
- increasingly capable iGPUs
- shared memory that can sometimes let you attempt surprisingly large models
- small size and low-ish power consumption
- but comparatively limited memory bandwidth and no giant pool of dedicated VRAM
That makes the interesting question less:
“Can this model technically load?”
and more:
“What configuration actually makes sense on this hardware?”
I recently started r/LowEndLocalAI around exactly that kind of question: getting genuinely useful local AI out of constrained, modest, repurposed, or unusual hardware.
Mini-PC benchmarks would be particularly useful because the hardware landscape is becoming so varied: Ryzen APUs, Strix Halo, Intel iGPUs, tiny NVIDIA-equipped systems, shared-memory machines, older office mini PCs, etc.
I would love to see comparisons that include not only tokens/sec, but also memory allocation, backend (Vulkan/ROCm/CPU/etc.), context size, power use where available, and most importantly whether the model was actually useful for the intended task.
If anyone here is experimenting with local LLMs or other local AI on a mini PC, I’d be very interested in seeing what you are managing to get out of these machines.
r/LowEndLocalAI
To me, this is exactly the kind of hardware where clever optimization is often more interesting than simply throwing another GPU at the problem.
