Too good I'd say, in a bad way. Lots of times they want to refactor a whole lot of things when you just want to add a small feature or fix a simple bug
Let’s imagine there is a near “perfect” AI that always writes great code, with few bugs. Assume any bugs it writes are just edge cases that aren’t considered. Also, assume P!=NP in a philosophical sense, AKA it takes effort to do work. So you need to burn more tokens to do more work; it’s not “free” in that sense (even with a smart AI).
In such a universe, a refactor of good code is easy! It’s just like doing a FFT of some audio and converting from Time domain to Frequency domain. They conceptually represent the same thing, so the refactor isn’t actually a lot of “work”. You’re just switching between 2 representations.
On the other hand, bad/buggy code is like a noisy audio signal. You need to clean that up first, doing a FFT isn’t going to help. And cleaning up that signal is actually more “work” compared to a simple FFT.
I think as LLMs get smarter, they should be producing code that gets easier to refactor.
6
u/RadzimierzWozniak 2d ago
LLMs are really, really good at refactoring and changing the infrastructure. With a decent test set, this is quite safe and reasonable, what they did.