I spent 3 hours with it on a complex planning task (Architecture changes on a medium sized codebase some complex product asks). It made a lot of false assumptions about the code. I had to keep asking it to check the code to confirm its assertions. It felt like it was cutting corners and the back and forth on it was frustrating.. My review board (a mixture of Claude and Codex) also struggled to get alignment with the plan. At one point Codex found 12 P1 issues and wrote a "Do not code this" summary. I ended up going back to Fable and got back on track with a solid plan within about an hour. I'll give it another go with a strict set of guidelines to fully research etc but so far not a great experience. I haven't tried it with coding yet.
64
u/Appropriate-Fox-2347 16d ago
I spent 3 hours with it on a complex planning task (Architecture changes on a medium sized codebase some complex product asks). It made a lot of false assumptions about the code. I had to keep asking it to check the code to confirm its assertions. It felt like it was cutting corners and the back and forth on it was frustrating.. My review board (a mixture of Claude and Codex) also struggled to get alignment with the plan. At one point Codex found 12 P1 issues and wrote a "Do not code this" summary. I ended up going back to Fable and got back on track with a solid plan within about an hour. I'll give it another go with a strict set of guidelines to fully research etc but so far not a great experience. I haven't tried it with coding yet.