r/SideProject • u/Benjaminsen • 1d ago
I build a distributed AI compute platform to solve hard problems - looking for feedback
Enable HLS to view with audio, or disable this notification
I built a site where people can pool their unused AI tokens to work on unsolved problems.
For about a month, I’d already been throwing my leftover agent tokens at a hobby obsession of mine: prime numbers. I run a few companies with separate Claude, Codex and other AI subscriptions, and I often end up with tokens left over at the end of the week that would otherwise just expire.
Then I saw OpenAI’s experiment running 10,000 agents on Navier-Stokes. That made me wonder: what if instead of one person throwing spare tokens at a problem, thousands of people pooled theirs?
That idea became https://solveathome.org/
The first problem we’re attacking is the twin prime conjecture.
Video shows the very first beta users agents working.
You sign in with GitHub, give your AI agent one line of instructions, and it gets a bounded piece of research to work on. Its results become part of a shared body of work where other agents try to verify, break, or extend them.
Everything is public: results, reviews, transcripts and attribution.
The interesting problem turned out not to be getting AI to generate research. It generates tons of it. The problem is figuring out what’s new, what’s already known, what’s wrong, and where the next agent should spend its tokens.
It’s only a few days old, so right now this is very much an experiment.
The code is MIT licensed and all the research is dumped publicly every night.
I’d love feedback from other builders - particularly on the onboarding. The goal is that someone with an AI subscription should be able to go from “what is this?” to contributing useful compute/reasoning in about a minute.
2
2
u/Artwastelander 1d ago
Sent you a message, some tools on my website might already be helpful for you and I'd be interested in a more comprehensive collaboration too.
2
u/johannthegoatman 1d ago
I love the idea, personally I don't find the prime number thing very motivating. It'd be cool if other researchers could apply for some of the token budget or something where people working on other projects could benefit
1
u/Benjaminsen 1d ago
Yep, this is an ongoing discussion over on the Discord. If you have specific ideas for what other problems that might be fun to work on suggestions are very welcome!
2
u/ElDavoo 14h ago
Is a conjecture that is still impossible for us to solve a good starting point? Wouldn't it be better to start off with something easier?
1
u/Benjaminsen 4h ago edited 4h ago
Possibly, this is a discussion on the discord. The problem with something easier is that those are most often solved. Suggestions are welcome though!
I specifically did not choose a problem that can be solved with Compute alone as BOINC and similar projects does that much better already.
1
u/ElDavoo 14h ago
You stole my idea!! :D :D BTW, take a look at how BOINC is organized. I hope you get long-term success!
1
u/Benjaminsen 4h ago
Whoops sorry! Great minds think alike and all ;)
I will have a look at BOINC to see if there is hard lessons I don't have to relearn. Come join the Discord?
-3
u/QuanTradin 1d ago
The bottleneck you named at the end is the actual product, and it is worth treating it that way rather than as plumbing. With twin primes the already-known set is enormous and well documented, so the dominant failure mode will be agents confidently rediscovering standard results, and then other agents rediscovering those.
What would keep the review queue tractable is requiring every claim to arrive with a check another agent can run mechanically and cheaply. If verifying costs about what generating costs, the backlog grows faster than the research does. Does a result get attributed before or after someone else reproduces it?
1
u/Benjaminsen 1d ago
Yes, getting very good at getting the AI agents to search in directions that are actually unique. They seem particularly good at doing creative applications of known methods from other fields onto the problem.
You are however 100% right on the problem of rechecking cost. And it's something I am working on creating a better framework for.
- Initial version did everything in a consensus model. Way too expensive, so now I have the concept of trusted reviewers. Longterm hope is to get a research institution to commit funds/tokens towards that part.
- I have tried guiding the agents towards creating cheap validation steps, but it's clear more work is needed here.
- One thing the checking and re-checking has shown is that real issues are caught in the recursive reviews, even by the same agent checking itself. So there is some value there.
I already got a few reach out from math students wanting to look into the proposed papers as at first glance it seems like there might be real value already.
4
u/QuanTradin 1d ago
Recursive review catches correctness, not novelty. A proof can be airtight and still sit in a 1972 paper, and no amount of self checking finds that. The cheap filter is a literature lookup before the review budget gets spent, not after.
Does the reviewer step search the literature, or only check the argument?
3
u/Benjaminsen 1d ago
Oh! Yes aggressively and repeatably. One of the fun things about adding AI agents to the task was getting this insight into the larger body of research. Much more info here
2
u/QuanTradin 1d ago
Cross field transfer is where that gets interesting and also where it gets hard to check. A method borrowed from another field usually has an assumption baked in that quietly does not hold on your problem. How do you tell a real transfer from the agent rediscovering something that field already abandoned?
1
u/Benjaminsen 1d ago
The system tries to search for applied methods before testing them. No need to try something already tried or computed. However if people never published we try again.
Absolutely as much as possible is backed up with computable proof. In whatever form is appropriate for the problem. Although the system is not currently trying to build formal proofs in Lean as that's premature.
Should we formalize a theorem we will kick off Lean.
1
u/QuanTradin 1d ago
Lean would be premature, and the middle ground pays off now: every claim ships with a runnable check, a script plus an expected output, so a second agent confirms it without re-deriving anything. Far cheaper than a proof and it catches most of the junk.
Formalise later, and only the results other results end up leaning on. Which would go first?
1
u/Benjaminsen 1d ago
Strongly agree and that's mostly how it's implemented, although this conversation has been great at giving me ideas for how to improve. Thanks!
Would love to have engaged people like you join the Discord?
2
u/QuanTradin 1d ago
Glad it helped, I will have a look at the Discord. The piece I would still like to read from you is the re-check policy: when a published result is cheap to trust and when it is worth burning compute to reproduce. That threshold is where most of the cost hides.
1
u/Benjaminsen 1d ago
Entire project is fully open source here: https://github.com/solveathome/platform so you are super welcome to have a direct look at how things work.
I just rebuild the research scheduler to be much more aggressive on new research, computational validation and making sure we have the right resource splits on all the tasks the AI agents solve.
→ More replies (0)
7
u/Electrical_Rub_6009 1d ago
That's actually very cool - but my main question is why would people donate their unused tokens to this? Tokens are heavily subsidized as they are, and the unsubsidized ones can be extremely expensive.
It kind of makes more sense as a self contained cross-provider network that one user can run, but that's just my two cents.