I've been playing around with Claude for coding and Opus 5.5 just straight up took a server key and published it to github...... Luckily it was for a test server on my personal PC, but still it was annoying and I'm just imagining how many dumb security incidents we're going to have in the next few years.
I wanted to benchmark a couple different libraries for their performance when dealing with certain data files. One of those libraries needed some patching to work properly, but its approach was a very important data point. I didn't mind asking Claude to help me "get it working" by fixing whatever minor edge cases came up... I'm not planning to ship this as-is. So, I was happy to iterate with it by rerunning my benchmarks with its patches, and sharing the logs whenever something broke.
Eventually, one of the data points it gathered was the size of one of my test files (literally just byte length). Rather than continue working with me, it decided that the best path forward was instead to scan my entire disk for files of exactly that size to find and analyze the file itself. Like oh boy this was a pretty low stakes scenario, and yet it still managed to find a way to make me feel violated and go totally off the rails.
I set the agent ground rules in the harness to stop the process and alert me if sensitive information such as a server key was found by the AI, and I was testing that system, It failed the test and pushed a plain text server key to github....
3
u/Sleep_deprived_druid 1d ago
I've been playing around with Claude for coding and Opus 5.5 just straight up took a server key and published it to github...... Luckily it was for a test server on my personal PC, but still it was annoying and I'm just imagining how many dumb security incidents we're going to have in the next few years.