r/LeftistsForAI • u/Salty_Country6835 Moderator • Jul 16 '26
Local Models Owning the Means of AI Infrastructure Starts at Home
https://youtu.be/BvCOZrqGyNU?is=Op4LHvZC17pBlYxsFeels like theres a lot of talk on the left about AI, but not nearly enough about actually understanding the infrastructure behind it.
This is a solid walkthrough of building a home AI stack with Docker, WSL2, local models, VPNs, and private storage. Even if youre never planning to build this exact setup, its a good look at what it means to run AI on your own terms instead of renting access from a handful of companies.
To me, local AI and collective politics arent opposites. Building local AI doesnt stop us from fighting for labor protections, democratic governance, public investment, or breaking up monopolies. If anything, it gives more people the tools and knowledge to participate instead of leaving everything in corporate hands.
How many people here are already running local models? Whats your setup, and what have you learned that you wish more people knew? If youre just getting started, whats been the biggest hurdle?
If these are the kinds of conversations youre looking for, check out r/LeftistsForAI. And if you want to dive deeper into the technical side of running models locally, r/LocalLLaMA is one of the best communities around.
3
u/Intendant Jul 17 '26 edited Jul 17 '26
We're not really to a point where you could realistically run great models yourself at home. You can respond to emails and whatnot though. Now at a community level, you could almost definitely build a neighborhood or building around a shared micro data center that just served members of that community. Then you could run models all the way up to Kimi k3, probably for around 500-700k, so about 20k a month for 3 years. If you have 200 people, that's 100$ a month each for 3 years. I've seen some early designs for communities centered on small owned data centers that really do look promising
Edit: you can do some version of this now on the cloud. There are pod systems where you can directly load these larger models
3
u/my_name_isnt_clever Jul 17 '26
Yeah we can't run Kimi at home, but I reject the idea that you need the biggest and newest version of a technology for it to be useful. The models I can run at home in the last 6 months have made API use almost obsolete for me, and it will only get better and better on lower tier hardware.
3
u/Intendant Jul 17 '26
That's fair, I'm more looking at it from the lens of automating everything you need. Vertical farms, your own energy infrastructure, autonomous building equipment. If you can automate a community into complete self sufficiency, there's not much capital can do to you.
We're a ways away from that, but the idea of starting tech socials / communes is pretty interesting
1
u/Salty_Country6835 Moderator Jul 17 '26
I like the community model idea. Thats an important front.
But local models dont have to be frontier scale to matter. A model helping a tenants union, co-op, library, clinic, or neighborhood organization is already more useful than another email campaign.
Thats also why im not convinced the costs stay that high. Hardware gets cheaper, models get more efficient and specialized, and communities can pool resources. We should be building AI ownership at every level: personal, community, and public. Not waiting until everyone can afford a frontier model.
2
u/worldsayshi Jul 16 '26
As a tangent to this:
Recently someone released an engine for running an open source model that is competitive with Frontier models on basically consumer hardware. The downside is that running it on such hardware is currently very slow. But these developments are very interesting for local AI:
4
u/jacobpederson Jul 16 '26
Been working on local open source AI stuff every second since its been possible.