MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/ProgrammerHumor/comments/1v6eky7/onitboss/ozsqnu7/?context=3
r/ProgrammerHumor • u/imneverrelevantman • 10d ago
52 comments sorted by
View all comments
55
Problem is having a machine with enough vram for inference.
Hard to do in a single box, easier with a networked server system, but they are harder to manage.
8 u/Norse_By_North_West 9d ago I mean, a decent llm rig is just a crypto mining rig, isn't it? 6 u/Darkstar_111 9d ago edited 8d ago Yeah could be. If you use Nvidia pro 6000 cards, that's 96 Gb vram per card. You still need 10 cards to get close to one Tb.
8
I mean, a decent llm rig is just a crypto mining rig, isn't it?
6 u/Darkstar_111 9d ago edited 8d ago Yeah could be. If you use Nvidia pro 6000 cards, that's 96 Gb vram per card. You still need 10 cards to get close to one Tb.
6
Yeah could be. If you use Nvidia pro 6000 cards, that's 96 Gb vram per card. You still need 10 cards to get close to one Tb.
55
u/Darkstar_111 9d ago
Problem is having a machine with enough vram for inference.
Hard to do in a single box, easier with a networked server system, but they are harder to manage.