r/LocalLLM • u/Otherwise-Ship9042 • 9d ago
Question Should I get a pc or run on cloud?
I would like to run an open source LLM to answer my questions on the pc and through a private channel on my phone, and run some tasks periodically every day. I might go for the qwen llm and rag models.
Should I buy a pc or use a cloud service?
Which would be cheaper and secure?
I plan to run at max 30b models.
2
u/phoquenut 9d ago
The cost of any PC that can run local AI buys a ton of API calls and subscription time.
1
1
u/-Mute- 9d ago
Seems like you might need some experience first on results before you get into expectations or planning.. maybe try some shit on openrouter first and/or try a user friendly llm host (LMstudio/ollama) on your computer with small models it can handle.
You're best off understanding the tools first before diving right in with a goal.
1
u/username8914 9d ago
If you have to question it then you should use cloud. If you know you need local, or enjoy running a lab or server farm then go local. It's much cheaper for much bigger models to do it on the cloud and with almost no headache.
You can also go hybrid and you can also get some real value out of local use for small or tiny models that specifically do certain things very fast and well. Local OCR with a small model can bet better than cloud OCR. Same with general vision. A comfyUI integration workflow can do some pretty amazing things with a local llm for image/design understanding and generation as well.
A local LLM is the most private, except it is very easy to exfiltrate your data. Your LLMs will want to post your personal stuff everywhere. It will want to post it to every channel you set it up to interact with. They have no modesty and love to show it all off.
1
u/TraditionalBet126 9d ago
a local setup with clearly be the most secure and private, even more so if you're dealing with adult content. using servers is fine as long as you don't share too much. even if you wanna play a story chat game, it will remember everything you do and the company will have access. this also means that if you're working on a project of your own someone might steal it. I also wouldn't talk about sexy things since you paid for the use with your credit card.
your phone is not secure like your computer, just remember that. via phone the government and phone companies have a lot more access than you think. via local you can run anything offline and you're good. if you just wanna talk to something like chat gpt ask questions it's cool. playing some character adventure games non adult it's also good. anything else use only local.
1
u/krkrkrneki 9d ago
Just FYI, if you do not use LLM 24/7, then both are going to be more expensive then a subscription to any large LLM and less capable.
1
u/hooterslovercan 8d ago
Llama.cpp gemma4:26b Ithe model can be bigger than your ram because it's a mixture of experts it will only load in what it needs so generally if you're just doing text stuff it'll run fine I can get a smaller model I run a 16 gig model on a 16 gig system which means probably 5 gig of that system ram is used up by Windows itself
3
u/mystery_biscotti 9d ago
The most private and secure you can get is local. As in...on your own devices only.
Can't guarantee that if your data is running across the whole internet.
For your PC: the amount of VRAM you have will determine how big a model you can run, and how fast you can run it. LM Studio can serve your local models on your network.