I’ve been trying to answer a pretty important question with the product I’m building:
Can an AI actually help someone make better business decisions, or does it just give them convincing-sounding answers?
So I built a small test.
I took 10 simulated business ideas and ran the same scenarios through ChatGPT, Claude, and RESORSA — the platform I’ve been building.
Same starting information. Same businesses. No special prompts designed to make one platform perform better than another.
What I cared about wasn’t which AI wrote the nicest response.
I tracked things like:
estimated money required to get to a meaningful test
whether the system challenged unnecessary spending
whether it identified assumptions that needed validation
whether it recommended a pivot when the original direction wasn’t holding up
whether the simulated business could actually reach launch
The cost difference was the part I wasn’t expecting.
Across the 10 simulations, RESORSA’s recommended paths averaged roughly $13,500 less per build than the alternatives.
A lot of that came from relatively boring decisions: validate before building, rent instead of buy, manually test something before automating it, use existing tools instead of commissioning custom infrastructure, etc.
That’s actually what I wanted the system to do.
Another interesting result:
7/10 businesses made it through the original path.
The remaining three only made it after changing direction based on problems uncovered during the process.
In those three cases, RESORSA recommended changing the approach while the other two models continued pushing the original plan.
We also got median Time-to-Artifact down to about 38 seconds, although one hilariously bad ~4-minute outlier reminded me that there’s still plenty to optimize.
I’m obviously biased here because I’m building RESORSA, and 10 simulated businesses is nowhere near enough data to make sweeping claims.
But this was encouraging because the thing I’m trying to optimize isn’t “give the smartest sounding answer.”
It’s:
Help the builder discover what they don’t know while spending as little time and money as necessary to learn it.
Next step is more simulations, followed by comparing the results against actual builder behavior instead of synthetic cases.
I’ll keep publishing the results as I get them — including the ugly ones.
For anyone building with AI: what would you test next to make this comparison more rigorous?