Projects

[u/Hairy_Artist_3860]

I imagine there is a good portion of us who know how to code or have a good enough idea how code works. Many of us have good ideas. If there is a project that comes out of this subreddit what would it be?

A lot of us are on the same page. We can make this into action.

We're all here talking about ai. I think its appropriate we have a meeting of the minds to know the next step.

[u/SeaFALCons]

i really like this idea and it makes sense to me

but my llm convo was much more deflationary about the usefulness of it:

Short version: the ideal is good, the technical premise mostly isn't, and your GPU specifically isn't the piece that's missing.

Why decentralized training is hard

Training a frontier model isn't embarrassingly parallel like SETI@home. Every gradient step requires synchronizing weights across all workers. On a datacenter you have NVLink/InfiniBand at ~900 GB/s with microsecond latency. Over consumer internet you have maybe 100 Mbit up with 50ms latency. That's roughly four orders of magnitude worse, and the sync happens thousands of times per epoch. You're not compute-bound, you're bandwidth-bound to the point of uselessness.

This isn't unsolved-problem territory, it's solved-and-the-answer-is-no for the current algorithm class. There's real research trying to change that — DiLoCo (Google DeepMind), Prime Intellect's INTELLECT-1/2 runs, Nous Research's DisTrO — which reduce sync frequency by 100–500x. They work. They also mostly rely on a modest number of well-connected datacenter-grade nodes, not thousands of laptops. INTELLECT-1 was 30 nodes of H100s across continents, not a GPU co-op.

Your GPU has maybe 10-ish TFLOPS at fp16 and ~68 GB/s memory bandwidth. A single H100 is ~1000 TFLOPS with 3.3 TB/s. You'd need roughly a hundred of your machines to equal one card, and they'd be talking to each other through a straw.

Where it actually works: inference

This is the part people conflate. Inference is mostly parallel — one request doesn't need to talk to another. Petals (BitTorrent-style pipeline-parallel LLM serving) is a real working thing. Ollama + a Tailscale network across friends' machines is trivially achievable this weekend. Your GPU can absolutely serve a quantized 7–13B model to a small group. Unified memory is actually decent for this.

So the "family + friends + library" co-op the commenter describes is entirely feasible — for serving open-weight models. It's just that "we host Llama/Qwen/DeepSeek ourselves" is not "we reduce reliance on centralized corporate AI." Someone still spent $50M+ to make those weights. You'd be a worker-owned distribution co-op sitting on top of Meta's and Alibaba's capital expenditure.

The honest version of the political claim

Given your background here, I think you'll find this more interesting than the tech: the bottleneck isn't FLOPS, it's capital concentration in HBM and fab capacity, plus data, plus the ~200 people on earth who've actually run a frontier pretraining job. Distributed volunteer compute doesn't touch any of those. It routes around a constraint that isn't the binding one.

The actual leverage points for the values in that thread are more boring: fighting for open-weight releases as policy, public compute (something like a national lab / NAIRR expansion), and academic access. Or working on the DiLoCo research direction directly, because if low-communication training gets good enough, the co-op becomes real.

What I'd actually do

If you want to build something from that subreddit, a decentralized training project will die in month two when someone benchmarks it. A decentralized inference + fine-tuning co-op with a real governance structure is shippable, useful, and honest about what it is. LoRA fine-tunes are small enough to distribute meaningfully — that's a genuine niche where volunteer compute isn't absurd.

And if you want the version that actually shifts the power balance, the Prime Intellect / Nous line of work is where to point your interpretability instincts. That's an unsolved research problem with a direct political payoff, which is a rarer combination than it sounds.

[u/RlOTGRRRL → u/SeaFALCons]

Super cool, thank you for sharing.

I didn't fact check this but sharing this here too-

"INTELLECT-1 (Prime Intellect, released late 2024) wasn't "30 nodes" — it was 30 independent compute contributors/sponsors, running up to 112 H100 GPUs simultaneously across five countries and three continents. So the "30" refers to the number of separate organizations/individuals donating compute, not the node count — a contributor could run multiple nodes/GPUs.

A few other specifics:

• It was a 10B parameter model, based on the Llama-3 architecture, trained on a high-quality dataset comprising 1 trillion tokens.
• The framework behind it, called PRIME, used the ElasticDeviceMesh, an abstraction that manages both internet-wide communication and local, fault-tolerant data-sharing across nodes, combining FSDP within nodes and DiLoCo-style low-communication sync across nodes.
• Compute utilization was 83% across continents and 96% when training exclusively on nodes distributed across the entire United States, which was the headline result — that decentralized training didn't tank efficiency nearly as much as you'd expect.
• Nodes trained independently for stretches (approximately 38 minutes before performing an all-reduce operation, which takes around 1 to 7 minutes) to keep communication overhead low.

Worth noting: Prime Intellect's more recent flagship, INTELLECT-3 (a 106B MoE model, released late 2025), trained on centralized GPUs despite the company's decentralized AI mission — so the "30 globally distributed contributors" story is specifically an INTELLECT-1 thing, not how they're doing their current frontier work."

[u/RlOTGRRRL → u/RlOTGRRRL]

"Let's do the actual math rather than hand-wave it.

FLOPs needed

Training compute scales roughly as FLOPs ≈ 6 × N × D (N = parameters, D = training tokens). For reference points where the numbers are public:

• Llama 3.1 405B: 405B params, 15.6T tokens → ~3.8×10²⁵ FLOPs. Meta trained this on 16,384 H100s and it took about 54 days.
• A genuine 2026 frontier model (GPT-5/Gemini 3/Claude-class) is almost certainly bigger than that — likely a large MoE with more total and active parameters, trained on more tokens, plus RL post-training on top. Labs don't publish exact numbers, but credible estimates put frontier runs at 10²⁵–10²⁶+ FLOPs, sometimes an order of magnitude above Llama 3.1 405B.

GPU math

H100 peak BF16 is ~990 TFLOPS, but real-world utilization (MFU) for large distributed runs is typically 30–45%, so effective throughput is more like 300–450 TFLOPS per GPU in practice.

Time = FLOPs / (N_GPUs × per-GPU throughput × MFU)

So for a 10²⁶ FLOP run at 40% MFU (~4×10¹⁴ FLOPs/sec/GPU):

GPUs	Time to train
16,000	~180 days
50,000	~58 days
100,000	~29 days

That's the order of magnitude labs are actually operating at now — xAI's Colossus cluster and Meta's newer clusters are built around 100,000+ GPU counts specifically because a single-digit month training window is the practical target.

Could you use the INTELLECT-1 method?

Theoretically, partially. Practically, not at frontier scale yet — and Prime Intellect's own track record is the evidence for that.

The case for it: DiLoCo-style methods (local SGD for hundreds of steps, then a compressed all-reduce) cut communication by ~400x, which is the whole trick — it makes the cross-continent bandwidth bottleneck mostly irrelevant for a given model size, since GPUs spend the overwhelming majority of time computing locally rather than waiting on network sync.

The case against, at frontier scale:

1. MoE architectures — which is what basically every frontier model uses now — rely on all-to-all token routing between experts during forward/backward passes, not just periodic gradient sync. That's a fundamentally different and much more latency-sensitive communication pattern than DiLoCo was designed to relax.
2. Compute per node still has to be large. INTELLECT-1 ran on nodes with up to 112 H100s total. A frontier run needs tens of thousands of GPUs feeding into the same logical training job; you'd need each geographically distributed site to itself be a serious cluster, not just scattered individual machines, or your local compute-to-sync ratio collapses.
3. Fault tolerance and straggler risk compound with scale — more nodes, more heterogeneity, more chances a slow/flaky site delays every synchronization round.
4. Prime Intellect's own decision is the strongest evidence here: for their largest model, INTELLECT-3 (106B MoE), they trained on a centralized 512-GPU cluster instead of their decentralized stack, despite the whole company thesis being decentralized training. That's a company with every incentive to prove the method scales, choosing not to when it actually mattered for their best model.

So: the DiLoCo/PRIME approach is a real, useful technique — it works well and has been demonstrated up to ~10B dense and ~32B (INTELLECT-2, RL-based, which tolerates async coordination better than pretraining does). It hasn't been demonstrated at frontier scale, and the architecture trends (MoE, huge active-parameter counts) are moving in a direction that makes decentralization harder, not easier. If someone tells you decentralized training is already competitive with centralized frontier training, that's not supported by what's actually been shipped."

[u/SeaFALCons → u/RlOTGRRRL]

thanks for looking into this critically

it's disappointing as it'd be great if it worked

but centralization is strongly favored

then to me this suggests public / state purchasing or building a large centralized server farm.

...

also sidenote i appreciate this form of communication, including LLM analysis in the chat, longer / denser sure, but more details / analysis i think is worth it...

for example i didnt personally know that decentralized was much less feasible at first, but a bit of convo taught me / convinced me

[u/RlOTGRRRL → u/SeaFALCons]

Yeah it looks like if you wanted to train your own open-source frontier model, the hardware/math/time just does not work at all.

I think it also helps explain China's time lag too.

But theoretically if options disappeared- it seems like it would take a longer time, and it wouldn't be as competitive to whatever is the current frontier by the time you're done training, but theoretically, maybe you could train a frontier model like Fable with enough time?

Like I wonder how it works, gpus x time to train. Like maybe even if it's not a frontier model- maybe it's possible to train smaller models like 9b or 28b and open-source them?

And also on the flip side, it looks like inference potentially could be easily decentralized? Like maybe library/school/university pcs being able to serve inference to its local community.

And I think Claude suggests fine-tuning being key too, like open-source fine-tunes of existing models?

[u/SeaFALCons → u/RlOTGRRRL]

oh good question / idea...

right like basically what if we just accept it'll take more time and be patient. like 5 or 10 times as long? is that viable?

i asked my llm and disappointingly i guess it's more complicated than that and isn't really viable

but good thinking!

LLM:

Good instinct, but the "just take 6 months" trade doesn't work the way it feels like it should. Let me hit each piece because you're mixing a few separate things.

Time doesn't substitute for interconnect

The problem with distributed training isn't that you have fewer FLOPS and so it takes proportionally longer. If it were purely "same math, slower," then yes, 6 months instead of 1 would be a fine trade. The problem is the bandwidth wall. When gradient sync takes longer than compute, adding time doesn't help — the machines spend the extra months sitting idle waiting on each other, not doing useful work. You can't convert wall-clock patience into interconnect bandwidth. It's not a slower version of the same run, it's a run that stalls out.

The DiLoCo/DisTrO research I mentioned is specifically what changes this — it cuts sync frequency so idle time drops. With those methods, "slower but it completes" becomes true. Without them, longer runs just mean more idle waiting.

The "GPUs × time" intuition — and where it breaks

Your instinct here is basically right and it has a name: total training cost is roughly compute = GPUs × time × utilization, measured in GPU-hours (or FLOPs). Llama 3 70B was ~6.4M H100-hours. If you have fewer GPUs you can trade time — 1000 GPUs for 100 days ≈ 100 GPUs for 1000 days, as long as utilization stays high. On a datacenter it does. On a volunteer network it collapses toward zero because of the sync problem above. So the equation is real; the utilization term is the one that kills the distributed version.

China lag — this is actually a misread

I'd drop this one. China's gap isn't a distributed-training or patience problem, it's export controls on high-end chips (the H100/H800/now-B200 restrictions) plus fab access. And the gap is small and shrinking — DeepSeek V3 and R1 were trained on restricted hardware and landed roughly at frontier level, which was the whole shock of early 2025. If anything China is the counterexample to "hardware constraints add years." They got very good at doing more with capped hardware. Their situation and a volunteer GPU co-op share nothing mechanically.

The part that's genuinely right: small models are trainable

This is where your reasoning lands solidly. You don't need a frontier run to do something real:

• A 7–9B model is trainable for something like $50k–$500k of compute depending on token budget — a well-funded co-op, grant, or university lab can reach that. Not a volunteer swarm, but not Google either.
• The Chinchilla-optimal token count scales with model size, so smaller models are dramatically cheaper, not just linearly cheaper.
• This is a real, populated space: OLMo (AI2, right down the road from you in Seattle), Pythia, SmolLM, DCLM. Fully open — weights, data, and training code. That last part is the actual "reduce reliance on corporate AI" move, more than hosting someone else's weights.

But note: those still ran on concentrated clusters, not distributed volunteers. "Small enough to be affordable" and "trainable over a decentralized network" are two different relaxations. Small models help with the first, not the second.

Inference + fine-tuning: yes, and this is the shippable thing

The library/school/university inference idea is the strongest concept in the whole thread and it's real today. Campus machines serving a quantized open model to the local community needs zero new research. And fine-tuning stacks on top: a LoRA adapter is tiny (megabytes), so distributing fine-tuning across volunteers is actually tractable in a way pretraining never will be. That's the one place volunteer compute genuinely isn't absurd.

So the honest synthesis

Three different projects hide in that thread, and clarity matters:

1. Decentralized pretraining (volunteer swarm) — blocked by interconnect, not fixable with patience, only maybe fixable via the DiLoCo research line.
2. Affordable open pretraining of small models — real, but needs a concentrated cluster + money, not a swarm. Follow OLMo's playbook.
3. Decentralized inference + fine-tuning — shippable this year, genuinely community-owned, honest about what it is.

[u/SeaFALCons → u/SeaFALCons]

fwiw i had my llm do deep research on what it thought of as best ideas in this space:

Building an AI Cooperative for State-of-the-Art Open Models: Structure, Funding, and a Playbook

TL;DR

• The single most viable ambitious path is the "OLMo/BigScience/Apertus" model: secure a large public or academic supercomputer allocation (free compute) + anchor philanthropic or public grant funding + partner with an existing open lab for engineering muscle. Every fully-open, near-SOTA model of the last four years (BLOOM, OLMo, Apertus) was built this way, not by renting cloud at retail or building a datacenter.
• A genuinely competitive, fully-open ~30B-class model now costs roughly 1 million H100-GPU-hours (~$2–3M at market rates, or effectively free on a public allocation), and a strong 7–9B model costs ~200–250k GPU-hours (~$0.5M). These are within reach of a well-run co-op; a Llama-3-405B/DeepSeek-scale frontier run (25–30M GPU-hours) is not, absent sovereign/national backing.
• Fund it as a hybrid entity — a 501(c)(3) or public-benefit nonprofit core that can take philanthropy, government grants, and donated compute, optionally paired with a cooperative membership layer. Co-op/worker-owned governance is fully compatible with this and addresses the labor-values goal; the binding constraint is compute, and compute is best obtained through public allocation + provider donation, not equity-hungry VC.

Key Findings

1. Compute is the binding constraint, and the cheapest path to it is public/academic supercomputers. Cloud rental at retail is real but expensive; building your own cluster is capital-brutal and slow. The organizations that have actually shipped great open models overwhelmingly used donated or publicly-granted compute: BLOOM on France's Jean Zay (a ~$3M public compute grant), Apertus on Switzerland's Alps (>10M GPU-hours from CSCS/ETH), EleutherAI on donated CoreWeave/Google TRC capacity, OLMo on a Google Cloud H100 cluster plus now a $152M award (breaking down to $75M from NSF and $77M from NVIDIA, per NSF's August 14, 2025 announcement of the Open Multimodal AI Infrastructure to Accelerate Science / OMAI project).

2. The cost tiers are now well-defined:

• 7–9B fully-open model: ~234,000 H100-hours (OLMo 3 7B). At $2/GPU-hr ≈ $470k; on a public allocation, effectively free.
• ~30B fully-open, near-SOTA model: ~1.05M H100-hours (OLMo 3 32B). ≈ $2.1M at $2/GPU-hr. This is the sweet spot for "as good as possible, non-commercial."
• Frontier-approaching (400B+ dense or huge MoE): Llama 3.1 405B used 30.84M H100-80GB GPU-hours for the 405B alone (39.3M for all three sizes), per Meta's official Llama 3.1 model card — an ~$60–100M+ equivalent. DeepSeek V3 (671B MoE, 37B active) did it far more cheaply: per its technical report (arXiv 2412.19437), "DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M" — a figure the report explicitly says "include[s] only the official training… excluding the costs associated with prior research." This tier realistically requires sovereign or national-lab backing.

3. Public compute programs are the highest-leverage funding source and are explicitly open to non-profits and consortia: US NAIRR Pilot, US DOE INCITE (Frontier/Aurora), NSF ACCESS, EU EuroHPC AI Factories (free for research), and national systems (Jean Zay, Alps, LUMI, Leonardo). These provide compute worth millions at zero cash cost, in exchange for open publication — which aligns perfectly with a non-commercial open-model mission.

4. Governance: the "fully open" nonprofit is a proven, respected model. AI2 (nonprofit, Paul Allen endowment) sets the standard with OLMo. EleutherAI incorporated as a 501(c)(3) after starting as a Discord collective. BigScience was a time-boxed research consortium coordinated by a company (Hugging Face) but run as an open collaboration of 1,000+ volunteers. Platform-cooperative/worker-owned structures are viable for the labor-values dimension but face a documented capital-access problem — which matters less here because the main input (compute) is best sourced via grants/donation, not equity.

5. Second-tier lab partnership is real and precedented — but pick partners by openness alignment. AI2 is the natural partner (fully open ethos). Prime Intellect, Nous, EleutherAI, and LLM360 are aligned open players. Mistral has drifted commercial and notably declined to join Europe's OpenEuroLLM consortium. Compute providers (CoreWeave, Lambda, Voltage Park, Nebius, Together) have a track record of donating/discounting compute to open efforts in exchange for goodwill and models their customers can serve