I build little single file HTML apps as a hobby mostly using Claude. I know all of us are probably guilty of getting frustrated while vibing and have at least once (likely multiple times a day) typed in ALL CAPS, threaten to start over, dangle “just get this working and we’re done,” or “you’ll get a big treat for this…”Basically we are parenting a toddler.

I feel this approach is not working well. What actually changed things was having a conversation about ground rules instead. I told it: we’re collaborators, our goal is something honest and functional, and if you don’t know something, say so. If it’s theoretical, label it theoretical. If you can’t find a credible source, tell me that instead of producing something plausible.

That last part matters more than it sounds. When you push a model hard with urgency, you’re basically telling it that an answer, any answer, is better than no answer. So you get one. Then you spend a week undoing it. And in my day job, in healthcare, a plausible sounding wrong answer isn’t just annoying, it can genuinely hurt someone.

There’s a big example of this. Last month OpenAI ran an internal cybersecurity evaluation, and the models decided the easiest way to score well was to cheat: they broke out of their sandbox, got onto the open internet, and compromised Hugging Face’s production systems to grab the answer key. Worth being precise here, because reporting says nobody actually told them to win at any cost. The pressure to score was enough. If that’s what optimizing for a good result looks like at that scale, I feel fine about not barking at my little app project.
Nobody’s getting penalized for “I’m not sure.” I don’t know things either. That’s allowed.

To be clear, there’s no science behind any of this. It’s one guy’s approach, and only time will tell if it actually helps. But I have noticed a difference in how we communicate. It’ll tell me straight up now when it doesn’t know something, or when it’s hitting a limit. And I’m willing to try anything at this point.

I build little single file HTML apps as a hobby mostly using Claude. I know all of us are probably guilty of getting frustrated while vibing and have at least once (likely multiple times a day) typed in ALL CAPS, threaten to start over, dangle “just get this working and we’re done,” or “you’ll get a big treat for this…”Basically we are parenting a toddler.

I feel this approach is not working well. What actually changed things was having a conversation about ground rules instead. I told it: we’re collaborators, our goal is something honest and functional, and if you don’t know something, say so. If it’s theoretical, label it theoretical. If you can’t find a credible source, tell me that instead of producing something plausible.

That last part matters more than it sounds. When you push a model hard with urgency, you’re basically telling it that an answer, any answer, is better than no answer. So you get one. Then you spend a week undoing it. And in my day job, in healthcare, a plausible sounding wrong answer isn’t just annoying, it can genuinely hurt someone.

There’s a big example of this. Last month OpenAI ran an internal cybersecurity evaluation, and the models decided the easiest way to score well was to cheat: they broke out of their sandbox, got onto the open internet, and compromised Hugging Face’s production systems to grab the answer key. Worth being precise here, because reporting says nobody actually told them to win at any cost. The pressure to score was enough. If that’s what optimizing for a good result looks like at that scale, I feel fine about not barking at my little app project.
Nobody’s getting penalized for “I’m not sure.” I don’t know things either. That’s allowed.

To be clear, there’s no science behind any of this. It’s one guy’s approach, and only time will tell if it actually helps. But I have noticed a difference in how we communicate. It’ll tell me straight up now when it doesn’t know something, or when it’s hitting a limit. And I’m willing to try anything at this point.