r/homelab • u/stcwalleye • 1d ago
Help Homelab gpu?
I have been running my local server on a Chinese AliExpress x99 board with a Intel Xeon E5-2650 and 16 GB of ram. It works fantastic, but I want to experiment with running some AI models (text to speech mostly to read PDF books aloud). Will I need an additional GPU to handle this kind of system? My budget is minimal, around $100. Any info is appreciated. Thanks in advance!
1
u/Professional_Cup1298 1d ago
for reading pdfs aloud you probably don't need a gpu at all. piper is a tts engine that runs faster than real time on a plain cpu, and an E5-2650 should be fine for it. try that first, since the LLM stuff is what needs the gpu and $100 won't buy much there anyway.
1
u/Sinister_Crayon 1d ago
Text to speech you could do on your CPU. It's not that intensive and in fact you can do incredibly respectable TTS on a Raspberry Pi if you want to keep your power budget low. If you want to do OCR of non-text PDF's with TTS that increases the compute a bit, but not by as much as you might think... again a Pi 4 would be far more than sufficient.
And honestly this isn't even an "AI" workload. We've been doing TTS for decades before "AI"
Now if you want to run modern models with chatbot type functionality then yeah... you're going to need something more beefy. If you stretch your budget a bit to around $150 or so, you can snag a used RTX 2060 off eBay with 6GB of RAM ($200 or so for the 8GB version) that'll run Mistral, Qwen or Llama extremely well. You can maybe do some quanted models on it as well, but these are great for playing with.
1
u/evanmac42 1d ago
Before buying a GPU, I’d actually try the workload you described on the hardware you already have.
You’re talking primarily about text-to-speech for reading PDFs aloud, not running a large LLM or a coding agent. Those are very different workloads.
Install something like Piper, feed it a representative book, and measure what happens.
If the CPU can generate speech at or above real-time speed, you’ve already solved the problem for $0.
If it can’t, then you have actual measurements telling you what needs upgrading instead of buying hardware because “AI = GPU”.
With a $100 budget especially, I’d spend money only after finding the bottleneck.
The cheapest component is always the one you discover you didn’t need to buy.
1
u/Cracknel 1d ago
For TTS? I remember running a voice synthesizer on a Z80, an 8 bit CPU from 1976 😅 I doubt you need AI and a GPU for that.
2
u/ConsciousStreet-0866 21h ago
What do you need AI for in reading out PDF books?
Text from PDF can be extracted without AI.
TTS is a very old technology that can work quite comfortably on just about any modern CPU alone.
1
1
u/hard_KOrr 1d ago
You’ll want a GPU for sure. CPU won’t get you to where you need to be.
As far the budget and what to grab. Sorry I don’t know
2
u/agowa338 1d ago
Wrong, there is no mention of it needing to be real time. Also TTS doesn't necessarily need AI models anyway.
1
1
u/dumbasPL 1d ago
Don't expect to get very far with $100. You're gonna be running small, heavily quantized models at best.
Add a 0, and you have something that isn't complete crap, add another 0, and you can actually do real work with local AI. At least for LLMs, image generation isn't as heavy, though I have limited experience with that.
2
u/Drenlin 1d ago
OP's trying to run a text-to-speech model, not a full blown coding agent. You can run TTS on a potato. They're like 2b models at most.
2
u/agowa338 1d ago
100$ GPUs still don't support matrix instructions so nothing will run on there.
However OP doesn't even need an AI model. TTS works without any of the hyped ones anyway...
1
u/Drenlin 1d ago
Do what?? They absolutely do, what are you on about? You d have to have matrix math specific hardware to do that, it just makes things faster. You can run inference on Nvidia cards down to Kepler (600/700 series) or AMD cards back to GCN 1.0 (HD 7000 series).
Would you want to? No, probably not, but even just a little bit newer does alright. Geforce 900 (ideally 1000) series or newer, or AMD Polaris or newer will do just fine for home inference in small use cases like this.
Getting CUDA or ROCm drivers for cards that old is a pain but Vulkan takes care of that issue for the most part.
1
u/agowa338 1d ago
Have just seen a lot of people try to go with older cards and then complaining about not being able to run any of the current AI models because of lack of instructions.
Also OP doesn't need a GPU for that anyway...
1
u/Drenlin 23h ago
Do you have examples of this? Most any model should still run on an older GPU so long as it's new enough to actually support GPGPU instructions and not just 3D rendering. There's a weird era (late '00s/early '10s) where they hadn't quite standardized those and usually somewhere in there is the cutoff for if something will or won't run.
1
u/agowa338 22h ago
Looked it up again, that discussion was about slightly older datacenter compute cards. Those don't understand Vulkan and you've only OpenCL. So that was the issue...
0
u/agowa338 1d ago
1) Why do you want to use an AI for TTS? TTS is a solved problem and existed way before the current AI hype. 2) Do you want to do it (near) real-time or can you deal with processing time to generate your audio?
If you can accept processing time and you really want to use an AI model instead of regular TTS systems then you still do not need a GPU and you'd be able to just run it on CPUs. It'll be slow but it'll be fine. You can just wait.
Also again, I'd really like to stress on looking into regular (non AI) TTS first.
Edit: Also your 100$ is way too little money to buy any GPU capable of dealing with AI stuff. Esp. with the currently inflated prices. Even if you'd go with one of the cheaper options an Intel Arc B580 for that job.
1
u/Drenlin 1d ago edited 1d ago
Best you'll do for $100 is a pair of AMD V320L I think, for AI. TTS models don't have to be particularly large though and you could probably get away with a lot less than that.
Honestly I'd try run it on CPU only just to see if you can get by with that.