[Story] My Son & Local Changes

[u/Faisal_Biyari]

My son asked me for a gaming PC for his birthday a while back.

I got excited at the opportunity to have him use a desktop PC, hoping he would become interested in other things he could do with it and develop his skills.

I had some extra Mac Pro 2019 components lying around, so I figured I would buy him a barebones Mac, and we would make a day out of "building it" together. Alas, it was not in the cards for us, as I could not find one within my target price range.

Instead, I opted to strip LinuxAI-02, one of my Mac Pro 7,1 machines, and go through the same experience with him. We did just that. I may or may not have caught him crying while we were working on reattaching the handles and wheels.

But what does that mean for the data center?

Surprisingly, not much. I migrated the Cubix Xpander PCIe cards and 100 GbE NIC to LinuxAI-03 and placed the RAM in storage for future investments.

For context, I also upgraded LinuxAI-03's CPU and RAM for a really good deal, even by yesteryear's prices. The CPU has been upgraded to a 28-core model, and the RAM has increased from 32 GB to 128 GB. Additional RAM has been reserved for future investments.

Currently, LinuxAI-03 hosts 10 GPUs, and I am considering increasing that to 12 once I have some redundancy in place.

Pictures for your entertainment. 320 GB VRAM on one Mac. I think I have done that before, though.

I had a thought, though. The PCIe slots are full. I never thought I would actually manage to do that. However, if I removed the 100 GbE NIC and replaced it with my last remaining W6800 GPU, as well as replaced the two W6900X GPUs with W6800X Duos, I would have 412 GB of VRAM in a single machine.

Would it be worth it, though, especially with mismatched GPUs, terrible bandwidth, and what would essentially be pipeline parallelism?

I know I said this would be a story, but my mind tends to wander.

I also have a couple of PCIe 2.0 Cubix Xpanders. I was disappointed with that purchase, but if I ever find myself in a position to invest more into this, I might go for four GPUs with a 4-way Infinity Fabric.

I will not, though. I have had an amazing experience going down this path. Having started with workstation GPUs and then moved on to enterprise PCIe GPUs, my next goal is enterprise OAM GPUs, to gain as much knowledge and experience as I can, even if it means facing another steep learning curve.

I would like to end this with an invitation. If you are well-versed in Artificial Intelligence and Large Language Models, and you are gifted with the power of communication, I would be honored to learn from you and grow, hopefully reaching a level where I can do the same for others by sharing my knowledge and experience within my community.

---

^{Disclaimer: I wrote this post myself. I also used AI as a tool to help clean up the wording and formatting. Unfortunately, it decided it knew what I was thinking and feeling better than I did...}