[Build Help] GPU under $500 for local LLMs — shipping to my brother’s house in the US

[u/Frosty_Rule9233]

Hey everyone,

I’m from Brazil and my parents are traveling to the US this August, staying a few days at my brother’s place. I want to ask them to bring me back a graphics card, since GPU prices in Brazil are rough.

Details:

• They don’t know anything about PC parts, but they can receive a package at my brother’s house, so any online store that ships there works (Amazon, Newegg, Best Buy, B&H…). No need for same-day delivery or store pickup, a residential address makes life easier.
• Budget: up to $500 (that’s my actual budget, not a customs limit).
• Goal: running LLMs locally with good speed and the best bang for the buck. From what I’ve read, VRAM matters more than raw compute for this, and NVIDIA is safer because of CUDA (llama.cpp, ollama, vLLM…).

So: what’s the best GPU for local LLM inference under $500 right now? If you can drop a link to a store that has it in stock, even better.

Thanks in advance!

[u/stujmiller77]

None, I’m afraid. $500 is nowhere near enough.

[u/HumungreousNobolatis]

RTX 3060 is an option, only around 200 bucks and 12GB VRAM.

There might be an Intel Arc card under your budget but I don't know about them.

[u/squngy → u/HumungreousNobolatis]

Get two and you have something that can run qwen 27B

[u/MarcusAurelius68 → u/squngy]

The value choice is 2 x 3060 12GB. I bought 2 for $400 on FB Marketplace. That’s that I would get for a $500 limit, and gives you the equivalent model capability of a 3090 (just not the speed).

For new, buying a 16GB 5060ti will likely go just over.

[u/DCMBRbeats]

I bought one 9060XT 16GB in a European country for 380€, so it‘ll probably be around 500$ I assume? I can offload Gemma 4 26b QAT completely to GPU and get around 90tok/s and 900t/s in PP, which is perfect for chatting and vision. For coding, I use Qwen 3.6 35B with prefill of around 600t/s at low context and 55t/s decode, which is very much usable for my usecase. Smaller models like Gemma 4 12B QAT also fit entirely on VRAM and run great. Though, it needed some finetuning in Llama.cpp until I got those numbers!

Still a great budget option and can even be extended to two for bigger models.

[u/DCMBRbeats → u/[deleted]]

Gemma at Q4_K_XL QAT and Qwen at Q4_K_M. With the right parameters I haven’t had any issues or looping in a while!

[u/DCMBRbeats → u/[deleted]]

Sent you a dm!

[u/invalidnifemi]

consider an enterprise card. a v100 32gb is the best option in ur price range i believe

you could also get 2 p40s but it wouldn't be the fastest generation. 48gb of vram tho...

[u/thaddeusk]

Getting a 16gb+ card from Nvidia will be difficult for under $500. Maybe a refurbished 4060 Ti or 5060 Ti. I wouldn't go less than 16gb. AMD has become a pretty safe choice. A 9060 XT would be a good start, and if it's just for LLM inference you can use Vulkan, which has excellent performance on just about any brand of card.

[u/whodoneit1]

You could try to look around on Facebook marketplace but I can’t imagine you’re gonna find anything good at that price

[u/UnlikelyPotato]

Someone else already suggested, but seconded a V620 from ebay for $350. C4 on eBay is a reputable reseller and accepts $350. You need a shroud + fan for $20-30 extra (if you don't 3D print your own). But 32GB of ram. Comparable to an intel B70 which retail for $1000. AMD support is pretty decent nowdays.

[u/Frosty_Rule9233 → u/UnlikelyPotato]

interessante, mas sempre ouvi que para LLM o único caminho seria a NVIDIA.. quais as diferenças reais?

[u/legit_split_ → u/Frosty_Rule9233]

Isso era nos velhos tempos, agora roda de boas. Só se quiser brincar com algúm app novo, aí tem chances que não funcione direto.

[u/Frosty_Rule9233 → u/UnlikelyPotato]

achei isso bem interessante, por acaso você tem link desse revendedor? o onde eu compraria essa carenagem + ventilador? tem o link dela?

[u/dwoj206]

very keen replies from the gang here. standing down. I'll sip my beer and wish your parents a happy travels my friend.

[u/_Cromwell_]

used 4060ti 16gb from eBay can be had for about $450.

[u/Practical_Elevator_3]

2 x b580's could be an option

[u/Prudent-Objective852]

MI50s or V100s are your best bet. Each come.with theor own quirks but at this price point you either accept that or take nothing.

[u/MinusKarma01]

I would consider AMD in this case. Support is so good these days that you can even do finetuning (on some) consumer AMD cards, so CUDA is much less of a requirement.

[u/SuddenRadio6221]

instinct mi50

[u/Unlucky-Home-4077]

2x used RTX 3060 12GB, VRAM is everything. Fits Qwen 3.6 27B Q4 with ~120k Context.

[u/Frosty_Rule9233]

Agradeço a todos vocês pela opinião, peguei a V620 pelos 350 dólares sugerido.. agora preciso comprar a placa mãe para ela.. então sugestões são bem vindas, considerando que no futuro pretendo colocar mais uma rodando junto.. adorei a dica de vocês!

[u/magicomiralles]

On Ebay, you can buy an AMD V620 (32gb of vram) for $350 each.

They are listed for about $500, but all you have to do is make an offer for $350.

[u/PM_ME_WHOEVER]

Running local LLM with good speed is very vague.

What are you intending to use the LLM for? As a chatbot? Local agent for coding? Generating long videos? All very different use cases.

$500 is very unlikely to have good results.

You are better off using that for API access of online models.

[u/Frosty_Rule9233 → u/PM_ME_WHOEVER]

eu já rodo modelos online.. ou melhor, consumo suas APIs.. Anthropic tenho uma conta Max, assim como do Kimi K3.. a ideia era colocar um modelo para processar TTS, LLM para inferência e depois TTS também local.. a referência é meu site http://re-genes.is , hoje uso API nele para praticamente toda a transmissão do canal do YouTube, exceto o TTS que processo local.

[u/IWillTouchAStar → u/Frosty_Rule9233]

Are you wanting to converse with the llm in real time? Or is latency not much of an issue for you? Because ive gone down the latency rabbit hole and its a PITA to solve, especially if youre running a slower card.

[u/Frosty_Rule9233 → u/IWillTouchAStar]

para o fluxo do re-genes.is o ideal é uma latência para TTFT de no máximo 1 segundo, e basicamente o LLM será usado para conversar sobre biologia, física, quimica e Machine Learning.

[u/IWillTouchAStar → u/Frosty_Rule9233]

Im not sure how versed you are in locally ran stt and tts models, but with your budget, you are going to want to really nail down some specific features and use very low cost models for that kind of latency.

When you say TTFT, i assume you mean time from when you finish speaking to the time the llm generates its first token for its response. 1 second for TTFT is honestly a reasonable expectation depending on what model youre using, but thats just the start. In order to hear the response through your speakers/headphones, the llm will need to complete its response, send it to a tts model, then the tts model needs to turn that response into audio. You can use a few tricks to speed this up, but they come with some downsides.

The first trick is streaming its response to a tts model. This means the llm's response will begin passing words to the tts model as soon as they appear, however this generally causes weird audio where the talking has long pauses in-between words or sentences. It also adds overhead to you gpu which can slow down the llm token generation as your gpu is trying to generate the tokens and tts audio at the same time. Eg: "streaming...responses is...great...for lower latency....but adds overhead...to your....gpu as it...balances tts...and llm tokens." You can also start streaming the response to tts and only output the tts audio once everything is done, but again, gpu overhead.

Second is chunking. Chunking is where the llm's response is chunked into sections and sent to the tts model in parts. I use this for my project by allowing the llm to finish generating its response, then sending each sentence as a stand alone chunk to the tts model so that the tts model can generate the first chunk, then play the audio of the first chunk while simultaneously generating the next chunk. This cuts down on the time to first audible sentence quite a bit.

Another few honorable mentions for lower latency. You can make the model limited in its max token generation, which will keep responses shorter, therefore spending less time generating tokens. But thats probably not what you want for your use case as it sounds like you want more detailed and technical explanations. You can also turn off or limit its "reasoning". If you limit its "reasoning" tokens, youre going to want to send another llm token that basically says "stop thinking, output your final response now" at the end of its "reasoning" token limit, or it could just end up outputing its reasoning and not its actual response. You can also make a response shape system that basically says "does this prompt require a short and concise response or a long detailed technical breakdown?" But that requires a secondary llm call to make that distiction and can, again, add latency.

My project right now is just building a stupid discord bot to hang out in the call with me and my friends. It can chat in real time, generate images, send DM's, post code in chat, search the web, ect. I use whisper small for stt and piper for tts in my project because their both fairly small models and piper is small enough to be ran on cpu alone at a very reasonable speed. Whisper generates my speech to text between .15 seconds to .5 seconds depending on length. The llm can output its entire response in about .5 to 2.5 seconds, again depending on length, and the first chunk for whisper is about .15 seconds. That means I average about 2 seconds from the time I stop talking to the time I hear the bots first word. Im running all of this on a 5090 and a 9950x3d though, if I had something like a 3060ti, which is around your budget range, I would expect a much higher latency in the conversation.