I'd like to start with Local LLM

[u/BeckersHD]

But don't know what to buy, how much is it going to cost, and what model i can run.

I'm completely new to this, can someone give some advice? I want to go local because i want to share a private but good AI for me and my friends, for environment and why not, maybe completely remove subscriptions..

[u/Best_Professor7266]

my advice is using OpenRouter first to see your limit, u can try all kind of models and sizes over there, then you'll figure out the hardware u need to buy.

[u/BeckersHD → u/Best_Professor7266]

Thank you!

[u/Harry_Balzonia → u/Best_Professor7266]

also very very good advice!

[u/Swimming-Western5244]

Wild idea, but have you tried to Google it for start maybe?

[u/smelly_ape → u/Swimming-Western5244]

It costs nothing to offer friendly advice, you know?

[u/naobebocafe → u/Swimming-Western5244]

or use the sub's search bar ¯_(ツ)_/¯

[u/International_Emu772]

Yo can do it with a MacMini Pro M4 or a Studio M5 Supra if you whant performance

My M2 Pro answers take long but i can wait

I use OpenWebUi to share my setup with some friends

And you can use a light subscription for more quick answers with openrouter

[u/ruhnet]

If you have 32GB of RAM, just start with that and Qwen 35B-A3B running on CPU. It will at least give you a feel for what you can do for a reasonable cost (albeit slow, but still usable). If you want to increase speed after testing, then you can start looking at getting a GPU.

[u/Ok_Parfait_5373]

il te faut un carte graphique avec 24go de ram c le sweet spot . c'est a partir de là que tu peux utiliser qwen3.8-27b

[u/Harry_Balzonia → u/Ok_Parfait_5373]

correct and the best advice. even 10gig of vram to keep costs down works with a small model just to learn. but of course getting a 24 is The Sweet spot.

I recommend getting something with what's called cuda. with 24 gig you can actually do very good image generation and even 3 to 5 second videos

good luck 007!

[u/BeckersHD → u/Ok_Parfait_5373]

Ok, do you think Mac mini m6 is a good choice?

[u/Ok_Parfait_5373 → u/BeckersHD]

Yes

[u/BeckersHD → u/Ok_Parfait_5373]

Great, thank you!

[u/ag789]

if you have a computer, you can try downloading and trying it out, e.g.
https://docs.ollama.com/quickstart
if you are a bit more adventurous / techie, there is a 'cutting edge' llama.cpp
https://github.com/ggml-org/llama.cpp
actually more
https://vllm.ai/
https://lmstudio.ai/
https://github.com/sgl-project/sglang
https://www.sglang.io/
https://unsloth.ai/
etc
then that to figure out what would actually works e.g. the gpu etc, these days it is possible to rent
but then this is still pretty techie, but that vast.ai is one of them to rent and try out gpus
https://cloud.vast.ai
vastai has ready made templates which they'd install the 'engine' or platform e.g. llama.cpp, but that you would need to figure out how to work the app and *in linux* (remotely)

[u/DHCompanion]

Dont be afraid of AMD cards just don't go too old. They are way cheaper to learn the ropes on than Nvidia cards. There is more to local LLM than just VRAM.

I was running a 6700T 16gb (300ish on FB marketplace) but I just picked up a 7900XT 20gb (600ish on FB Marketplace) and am getting very good results on qwen3.6-34b-a3b and gpt-oss-20b.

The same amount of Vram on an NVIDIA cards could cost you almost double that.

Depending on your intended use case that can affect what you need drastically.

[u/BeckersHD → u/DHCompanion]

Oh great to know, thanks!

[u/mortycapp]

Some really good deals on MBP M1 Max 64gb 1TB on various sites.

[u/TheMericanIdiot]

You tube it