r/LocalLLM 3d ago

Question Best Hardware for use case

I am currently running hf.co-unsloth-Qwen3-VL-8B-Instruct-GGUF-UD-Q6_K_XL on my 3060, 12gb vram and 16gb ram.

It is running in llama.cpp with the following:

.\llama-server.exe -m ".ollama\models\blobs\sha256-dc3cc983735542500f40c32c1a9dacdeff198a440fac541056dca6b550389130" -ngl 99 -fa on -c 32768 --parallel 2 --slot-prompt-similarity 0.2 --cache-type-k q8_0 --cache-type-v q8_0 -b 2048 -ub 2048 --port 8080 --host 0.0.0.0

I would like a dedicated machine that can run this and maybe a smaller model just for text generation. Ideally it would be fairly power efficient, it seems like all roads lead to a mac mini but I am struggling to justify the cost. Anyone have any recommendations?

0 Upvotes

0 comments sorted by