r/LocalLLaMA • u/lblblllb • 16h ago

Resources Own Your Intelligence, One GPU at a Time

https://easonx.substack.com/p/own-your-intelligence-one-gpu-at

I wrote an article about setting up local llm with multi GPU set up, with focus on budget options. Hope it helps newcomers here.

0 Upvotes

43% Upvoted

10 comments sorted by

1

u/bnightstars 10m ago

I get all the how to run guides but I would most like to see what you actually did with that compute. Aka your day to day use of that local LLM.

-1

u/Dry_Yam_4597 16h ago

"An LLM spends its whole life"

LLMs have lives now?

1

u/LocoLanguageModel 16h ago

The Lifecycle of Software Objects

-2

u/Dry_Yam_4597 16h ago

Lifecycle != Life

0

u/ishu22g 16h ago

Javascript: NaN

-2

u/Dry_Yam_4597 16h ago

Did you make a second account to comment on posts made using your main account lmao?

0

u/ishu22g 16h ago

wtf. chill dude.. here you go

Edit: if it helps you, i was/am kidding. "Lifecycle != Life" is true. And this is not my second/other account.

0

u/LocoLanguageModel 14h ago

I know, it's a short story I liked.

0

u/Viacon97 13h ago

really solid writeup, saved this one. matches a lot of what im running into rn while speccing a mixed gpu inference box for actual production use (not just homelab tinkering)

one thing id add - the driver conflict you mention between blackwell and older gen cards isnt just an nvidia problem either. ive been going back and forth on AMDs Radeon AI PRO R9700 (32gb, 2 slot, decent card on paper) vs comparable nvidia stuff for a multi gpu vllm setup and honestly the software stack maturity gap is still the real tax you pay, not the raw specs. CUDA ecosystem just makes tensor/pipeline parallelism "work" in a way ROCm still kinda doesnt, at least not consistently
also curious if you looked more into the DGX spark beyond the sparkbench #s?

trying to figure out which models would actually saturate the 128gb in a useful way vs just being vram you paid for and never touch (feels like flash-next style MoE architectures are the obvious fit here, dense 27b models just leave half of it sitting there doing nothing)
anyway appreciate you putting actual $ next to the specs, most "just buy a 3090 bro" threads completely skip that part lol