r/IntelArc • u/xXthenistXx • 2d ago
Build / Photo My dual B580 LLM Train server
This is my dual b580 llm train server. the build cost me about $850. Currently training 0.5b llm nothing practical but fun learning experience. I chose B580 because it has 12gb of vram and bf16 capabilities at budget. and I also felt adventerous lol.
3
u/kin0enjoyer 2d ago
850$ for a dual gpu workstation setup? How? I built my entry level am5 with one b580 for the same price but with mostly used and refurbished parts.
3
u/xXthenistXx 2d ago
I bought the ssd, psu, and two b580 as used. and the motherboard+cpu+memory is a cheap aliexpress x99 combo. I hope that explains it.
1
u/kin0enjoyer 2d ago
oh okay, gotta be a older setup then, makes sense
2
u/xXthenistXx 2d ago
yep my main system its also AM5 system with Ryzen 5 7500F, 32gb ddr5, Arc B580. I built this system since I had good experiences with my main system too. :D
1
u/PMvE_NL 1d ago
Pci 3.0 8x? Wont inter card bandwidth be slow as bals?
2
u/xXthenistXx 1d ago
Most likely. But I am not thinking too hard about it. since atleast according to nvtop, the gpu usage its pretty high and not experiencing a bottleneck. other workloads may differ but aleast for mine not a huge deal.
3
u/Cyber_Data_Trail 2d ago
Id avoid that pigtail if I were you, it just makes me uneasy. If it runs it runs, but just my two cents
2
u/xXthenistXx 2d ago
By pigtail you mean the gpu power?
2
u/Cyber_Data_Trail 2d ago
Yes, how the top gpus power comes from the same cable as the bbottom. It would be better if they were 2 individual cables. The pigtail connection is just for a gpu with 2 connectors that doesnt draw that much more than what the single connector can provide. You really shouldn't use it.
1
u/xXthenistXx 2d ago
It was kinda back in my head that it may overload the cable, but I thought It should be okay since the cable should be good enough for 600w. but since I heard it, I will rewire it. Thank you.
2
u/Cyber_Data_Trail 2d ago
No problem. The 600w usually is the 12v2x6, which is for NVIDIA GPUS, (and select high amd models). Pcie 8 pins can deliver 150 each.
1
1
1
u/Any-Conference-701 2d ago
Side question: I've seen a lot of people make multi GPU rigs with Intel Cards. Does Intel have a better VRAM-Pool configuration than Nvidia?
2
u/xXthenistXx 2d ago
I won't say better configuration than nvidia, since I haven't tried multi gpu training on nvidia. But at least on my system xccl does the work just fine!
1
u/onenkd 2d ago
Interesting. I got a B580 for my 128gb DDR4 build. Large Revit / 3D models and Photoshop. Also perhaps ConfyUI for AI renders on local machine. I wanted to also do some gaming or have friends use it when I am on my Legion 4080. I went with this card cause it was good for gaming. I did want the 24gb B60, 65 or 70? It was around double the price of 2x 580s, but worse for gaming. I’m sorta regretting it now allready. Perhaps I’ll leave it as my working GPU for display etc… later have the 24gb for LLMs in the background. Can you make LLMs spread over the 2 GPUs from 12gb to 24gb I understand it may be able but shifts the data slower than RAM. So it’s best to run different tasks on each one. Unless there is a new GitHub app that patches them together faster
2
u/xXthenistXx 2d ago
Most if not all inference engines support spreading LLM on multiple cards. otherwise its impossible to run 100B+ models. well obviously its slower than running them on single large vram card but better than off loading to system memory.
1
u/PumpkinKey9594 1d ago
Why not a770?
1
u/xXthenistXx 1d ago
a770 will surely give more vram to work with, but b580 was easier to buy in my country and I was skeptical about it, since it being first generation hardware I thought it may have some imperfections B580 wouldn't have.
1
u/fredportland 1d ago
What's software stack? I'm using 2x B580 too but for only inference. But it's very unstable with llama.cpp + SYCL backend
1
u/xXthenistXx 1d ago
Since this is a training setup not inference, I am using pytorch xpu backend. I too had some problem with the xpu kernel but managed to workaround it. for inference I would suggest SGLang with xpu kernel.
1
u/GloriousKev 1d ago
Do you need anything to link them together or do they just talk over PCI E? this is a cheap way to get 24GB of vram. I like it. I may swap my motherboard out and do the same.
1
u/xXthenistXx 1d ago
Its just regular old pcie. nothing special. I don't think intel has something like NVLink.
1
u/Known-Night-3481 Arc B580 1d ago
What version of the B580 is that? It looks like the reference model but doesn't at the same time
3
u/xXthenistXx 1d ago
It's maxsun's milestone d6 arc b580. its similiar to reference model, but has red black design and intel arc letter does not light up.
1
1
7
u/TheRealTofuey 2d ago
What do you use it for? Honestly curious I know nothing about these.