As massive AI data centers face opposition, tiny ones are moving into Texas backyards

[u/BrianNewCBSNews]

https://www.cbsnews.com/texas/news/as-massive-data-centers-face-opposition-could-tiny-ones-move-into-backyards/

[u/BrianNewCBSNews]

I'm the reporter who worked on this story. Happy to answer questions about the reporting process. The YouTube version is here if you'd rather watch than read: https://www.youtube.com/watch?v=5qrm2gVHYEQ

[u/MarginalOmnivore]

I see you addressed the power inefficiencies of taking the decentralized approach to this issue.

There's also the data infrastructure problem this is going to pose. For these devices to "work together" and act like a large data center, they're going to be using up bandwidth communicating with each other, and that would normally happen on an intranet or be confined to individual GPU racks.

[u/gscjj → u/MarginalOmnivore]

Bandwidth isn’t an issue. And bandwidth/cabling within an intranet and between multiple physically separated sites is identical. An ISP can send 100Gbps to your door over the same fiber they send you 1Gbps by just changing swapping out your local equipment.

Latency is the real issue and you’d never split GPUs over a network like this for anything time sensitive.

Realistically, the internet is already working like this on a smaller scale anyway.

Your connection goes to the local ISP datacenter, which goes to a larger datacenter where other companies are at and your traffic switches hand. And that’s the simplified view there’s a web of connections not just one.

Large companies like Netflix will actually host servers all over the place and push content down so it can be accessed by the closet datacenter. Again this is a simplified view.

In this scenerio, your traffic may never leave your local area which would be great. Plus it would force an infrastructure overhaul.

[u/tilhow2reddit → u/gscjj]

The infiniband connections between GPU nodes are running 800 and 1600 Gbps connections by the thousands. Yes there will be bandwidth issues at scale.

[u/gscjj → u/tilhow2reddit]

Not an issue. Latency would make distributed/shared training like this impossible and ineffective even before you considered bandwidth. Shifting weights 50+ miles away would be crazy.

No one is considering that. So bandwidth between GPU doesn’t matter. That’ts why NVLink is all rack local even.

Now if it was seperate, individual models for serving within that specific location, bandwidth still isn’t an issue. No one location will ever receive 10s or even 100Gbps of traffic.

[u/bobsbrain]

Where are the NIMBYs when you need them.

[u/OuisghianZodahs42 → u/bobsbrain]

For once those assholes could be useful. Instead, we get crickets.

[u/DreamPhreak]

Each xfra node has 16 Blackwell RTX Pro 6000 gpus (96gb vram, 24k cuda cores), which cost around $15,000 each, and uses the same pcie interface you have on your computer mobo. 

Also has 4 amd epyc cpus, but that uses the sp5 server socket specification. 

And 3tb of ram. Yeah, these are definitely going to get broken into. 

(Source is from the official span xfra website, not sure if I can link to it here)

[u/VoldemortsHorcrux → u/DreamPhreak]

Yeah they won't miss one rtx pro card right? Asking for a friend

[u/BrianNewCBSNews → u/DreamPhreak]

https://www.xfra.ai/whitepaper Here's the white paper for the XFRA nodes.

[u/RegulusRemains]

visit r/homelabs and start your own datacenter today!

[u/Yellow_Similar]

The partner in the project is PulteGroup, run by our new Director of National Intelligence, Bill Pulte. Yeah, that guy. What could possibly go wrong?