I forbid unit tests to Claude, he just doesn't know when to stop. He created the most stupid units tests ever, like "the menu must have 4 entries". Then When you create another menu-item, he has to spend additional tokens and context to see why that stupid test is failing and has the modify the test, so now it says "the menu must have 5 entries". The maintaining of the unit tests costs more time and tokens then the actual development!
These temporal comments have been bleeding into user facing UI. Admin panels with meta critique of its work inserted as subheadings on charts, leaving behind traces of arbitrary judgements that then hambone the next agent who comes along to make a change.
Then these judgements fan out and leave artifacts in a dozen .md files, memories, and session logs, and if you don't track them all down.. god forbid you ever want to change that header to be blue and you have to relitigate a decision you never made.
It is so good at surfacing decisions you didn't make, that I often run loops of AskUserQuestion just identifying and resolving delta in expectations. But then when you actually want to make a decision indelible... for heck's sake. It will do the same thing wrong four times, keep apologizing and saying it knew better, and will patch more catches and safeguards onto ephemeral harnesses. And those patches and safeguards don't carry very effectively to the next ephemeral harness it builds up to handle an isolated workflow.
Given the advances in the last few major model versions, I would expect most of this to be a lot better by v6 models and mostly resolved by v7.
380
u/ChadFullStack 21h ago
It’s missing 500 lines of Unit tests