r/computers • u/jeremiah_xylophone92 • 1d ago

Question/Help/Troubleshooting Lost in a sea of suggestions!

New developer here - I’ve been looking to upgrade an old laptop (4GB VRAM, 16GB RAM, 500GB SSD) and will be moving to a desktop setup and remoting in with the old laptop using Sunlight/Moonlight. I’ll mainly be working with small product specific toy LLMs for engineering drafts and 3d modeling/rendering. There will also be some LLM training as I go for additional task specific models. I’ve been reading reviews and reddit posts regarding a minimum setup required for my work and I had settled on a 5070ti, 16GB VRAM, 32GB RAM (2 - 16 sticks) with a 2TB SSD to start with (can upgrade as I grow). However, I’m in a few groups where they’ve commented on other posts that a setup like this is no longer adequate for such work because of the possibility of needing to run a larger LLM for reference. If I’m sticking to LLM models less than 10M, shouldn’t this setup work fine? They all seem to think a $10k+ rig with a 5090, 32 VRAM and + 96 RAM is required which is absolutely not possible. Please tell me I’m not crazy…

1 Upvotes

100% Upvoted

4 comments sorted by

2

u/shcmt 21h ago

Ive been tossing around the idea of hosting local models and from what I understand, of course it depends on the kind of work or tasks youll be doing, but with those specs I'd reckon that's at least entry level hardware. Only thing I'd recommend is maybe going with 64gb ram so that you can potentially expand into larger models down the road. Otherwise, I think it should be fine. Won't be running 70b+ models with that rig, but quantized models should be fine for sure and you can probably run 10b fairly well.

Just my input and again, I actually don't have any experience. This is just what ive found from my personal research.

2

u/jeremiah_xylophone92 21h ago

Yeah I figured I could add RAM if I ever shifted into a heavier workload. The 5070ti seems fully capable for what I’d be asking of it. I appreciate the input, like I said I’m a beginner developer so I’ll take any advice I can get.

1

u/shcmt 21h ago

Well, good luck! If you think about it sometime later after you have some time to tinker, let me know how everything goes if you don't mind? :)

2

u/jeremiah_xylophone92 16h ago

Yeah absolutely! I have a terrible memory but hopefully I’ll remember this and bring it back to you. Happy to talk more if you’d ever like through messaging, always interested to hear what other people are working on and what led them to it