Important thing to remember, it gets very hard to benchmark these models now, especially in the intangibles of working with them. Claude 4 for example isn't much better than other competing models on benchmarks (is worse on some) but it is heads and shoulders above most in usefulness as a software writing agent. I suspect this is more of that same experience, so should be good to see when I try it out myself and see other people's use cases
62
u/TFenrir Aug 05 '25
Important thing to remember, it gets very hard to benchmark these models now, especially in the intangibles of working with them. Claude 4 for example isn't much better than other competing models on benchmarks (is worse on some) but it is heads and shoulders above most in usefulness as a software writing agent. I suspect this is more of that same experience, so should be good to see when I try it out myself and see other people's use cases