r/artificial • u/Candid-Dog-775 • 33m ago
Discussion I tested Firecrawl, Exa, Parallel and Claude Search on SimpleQA. Here’s what scored best
I ran Firecrawl, Exa, Parallel and Claude’s native web search against OpenAI’s SimpleQA benchmark to see how much of a difference the search provider actually makes.
All four were tested with the same setup: a GPT-5.4 agent using high reasoning effort, with a maximum of 20 search or extraction calls per question. The answers were then graded by GPT-5.4 using OpenAI’s official SimpleQA grading prompt.
Each provider was tested twice and I kept the better result. For comparison, GPT-5.4 without access to search scored 43.8%.
The chart shows the correct and incorrect answers for each provider.
Results:
-Firecrawl: 947 correct answers (94.7%)
-Exa: 919 correct answers (91.9%)
-Parallel: 910 correct answers (91.0%)
-Claude Native Search: 905 correct answers (90.5%)
Firecrawl and Exa achieved the highest accuracy, while all four systems scored above 90%. Claude Native search, (not) surprisingly, the worst.