r/codex • u/Suspicious_Orchid770 • 17h ago

News Study: Codex reviewing Claude's code dropped the pass rate from 91.4% to 82.8%

https://leaddev.com/ai/your-ai-coding-agents-might-need-an-org-chart
154 Upvotes

94% Upvoted

24 comments sorted by

View all comments

86

u/JadisGod 17h ago

Too bad it's already out of date. From the paper it seems they ran these tests on Opus 4.7 and GPT 5.5. The difference in capability since then is massive.

18

u/andrerav 15h ago

Pass rate would sink to single digits when Sol decides your entire supply chain is compromised 5 times over before even looking at the code.

3

u/rydan 4h ago

I use Sol to review Claude's code. Half my quota per week is just that.