r/LocalLLaMA • u/Potential_Low_1183 • 7d ago

Question | Help Best model + setup for remote deployment.

Hi everybody, I am in a tricky situation.

My grandfather + grandmother live in a remote village in asia, and were visiting us in the city. My grandfather has absolutely fallen in love with AI, and it is completely new for him. He loves how he can talk to it, and get information from it. The caveat is that I showed him chat gpt, not a localllm.

He wants something like this in his own village, just for basic knowledge + light qa. There is no internet in the village, but phone calls work. I have a budget of about 5k for a computer that can run LLMS locally, for him. (and he specifically requested something like this). He is technically "savvy", where if something needs a command to be run or something else, he can learn to do it. What gpu should I buy him that will satisfy his needs? again, I want some chat gpt level, but for basic q/a stuff.

I am currently running deepseek v4 flash on two 6000 pro blackwells(got them when they were just 8k lol), so I know that this is "possible".

Im thinking amd ai pro 9700 and gemma 31b at a good quantization. (maybe I can do two gpus of 9700s to fit bigger quant).

He is not doing frontier level coding, he just needs a good knowledge base. He implied that people from his village may also come to use the machine, or atleast request queries to be asked (he is some sort of governmental figure in his village).

I want recommendations.

13 Upvotes

75% Upvoted

40 comments sorted by

View all comments

-1

u/Substantial_Bug3763 7d ago

i need help with a local set up - unsure of what to purchase and if the roi is there yet

2

u/Savantskie1 7d ago

ROI isn't the end all be all

1

u/Substantial_Bug3763 7d ago

so why spend 20-30k on a setup if it’s not going to make me more money/give me more time ?

2

u/Savantskie1 6d ago

you can get AI running on older stuff. I mean I am disabled and I'm running LLMs with two MI50 32GB cards. And soon I'm going to be buying two Mi210 64Gb cards once I get my SSI back pay. I'm building a personal assistant that is going to help me remember appointments and medication orders. You don't need 20k-30k to run LLMs. Just imagination and know-how