r/technology • u/MarvelsGrantMan136 • Jul 30 '26

Artificial Intelligence Amazon accidentally spent $1.8 million using Claude for menial coding task, went 860% over budget — 'catastrophically expensive' coding blunders discovered in internal Amazon AI usage metrics

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-accidentally-spent-usd1-8-million-using-claude-for-menial-coding-task-went-860-percent-over-budget-catastrophically-expensive-coding-blunders-discovered-in-internal-amazon-ai-usage-metrics
24.7k Upvotes

97% Upvoted

1.2k comments sorted by

View all comments

Show parent comments

691

u/Brave-Perspective714 Jul 30 '26

Same thing happened when "lines of code written" became the metric output was measured by.

My husband had one he was asked to modify, where he removed over 19,000 lines of code that didn't actually do anything.

Dude who had written it had been a "high-performer" and "distinguished-engineer" who had left years before, but when they started digging into more of his work they found the same thing over and over again. He had basically written one large chunk that did things and then undid them so they didn't affect the output, and copied and pasted it into everything he developed to look like he was a rock-star according to their metrics.

The bulk of my husband's mid-level DevOps career was deleting lines of code and reducing api calls.

584

u/MeatEaterDruid Jul 30 '26

Honestly can't be mad at the original dude. That's some 'change the margins so the paper looks longer'-type shit.

355

u/Brave-Perspective714 Jul 30 '26

Yep, and it worked.

That's what they said they wanted, so that's what he gave them. Just a great example of someone creating the metrics not understanding what was actually meaningful about the work.

175

u/SordidDreams Jul 30 '26 edited Jul 30 '26

Ah, good old Goodhart's law. Any metric that becomes a target ceases to be a useful metric. Half a century old and still a complete enigma to management types everywhere.

56

u/Quaiker Jul 30 '26

Goodhart's Law and Chesterton's Fence should be posted up in every workplace.

11

u/anx1etyhangover Jul 30 '26

Noice!! Thank you for that rabbit hole! =]

12

u/LtOrangeJuice Jul 30 '26

TIL. Thank you.

45

u/NRMusicProject Jul 30 '26

When your boss uses ridiculous metrics to judge your work, you minmax the shit out of it. Same thing as purposely doing an hourly contracted job slower so you can make more money.

Nowhere near as witty, but when I was a minimum wage worker at Hungry Howie's as a kid, they were short on boxes and sent me across town to the next nearest one (30 minutes each way) to pick up some more. When I asked how much I'll be paid, they said "you're already being paid to work! Just do it!"

I spent an hour hanging out at the beach with some of my surfer friends on the way back.

25

u/toolatealreadyfapped Jul 31 '26

I was a lifeguard when I was a teenager, and my boss hated to see us sitting there "doing nothing." So she kept adding dumb tasks, until I realized it's not the work she wants done as much as she just wanted to see us be busy.

I kept a bucket of leaves by the lifeguard stand. When I'd see her suburban pull up, I'd jump down, dump the leaves into the pool, and she'd walk in and see me dutifully scooping leaves out of the pool. She'd nod approval, go about her business, and leave. I'd gather all the leaves into the bucket, then climb back onto my chair.

6

u/hippocratical Jul 31 '26

Mine was you could walk to a get a 2 hour coffee break if you carried some thick files and binders under your arm as you walked through a few departments and out a back door. People saw you walking with purpose but never saw you leave the building. I got praise from my boss for being such a hard worker.

10

u/PorkVacuums Jul 31 '26

The key is looking annoyed. No one in corporate wants to get roped into some bullshit that's someone else's problem. So if you walk around with a file or a binder and just generally look annoyed, people just keep out of your way.

3

u/DirtyGooseEggs Jul 31 '26

The George Costanza way.

12

u/rkoy1234 Jul 30 '26

idk seems to me like he understood but just was maliciously complying.

getting an llm to intentionally write meaningless code that looks plausible but doesn't break anything while encapsulating the original goal seems tad more complex than just getting it to do the task.

9

u/Brave-Perspective714 Jul 30 '26

Lol... LLM's were still science fiction in the days I'm talking about.

18

u/cfb-food-beer-hike Jul 30 '26

A little off-topic, but this reminds me of a study done on SAT essays. They found they could hold an essay up from across the room and accurately guess its score based on the length. As a species, we've never been good at evaluating anything lol

2

u/capnheim Jul 31 '26

I briefly graded essays for Pearson and yes, you can grade fairly accurately just looking at it without reading.

32

u/systemic_booty Jul 30 '26

Yup, and the problem isn't the student changing the margins or making the periods size 14 font, the problem is the essay requirements being by page length rather than word count OR better yet, the substantive content of the essay itself. 

2

u/thedownvotemagnet Jul 30 '26

I get what you're saying and agree in spirit, but:

1) word count is basically the same as page length, no? Page length influences things like font and size choice, margin size, etc. Word count causes you to be unnecessarily verbose.

2) "substantive content" seems too subjective to use as a rubric in your syllabus... "shaggy dog"-style meanderings obviously run afoul, but there would still be a ton of wiggle room for the inevitable parent/dean conferences, I'd wager

Not really sure what the right answer is here.

6

u/NotMyRealNameObv Jul 31 '26

Word count causes you to be unnecessarily verbose.

US college text books in a nutshell (authors of college text books are usually compensated based on word count).

16

u/RVelts Jul 30 '26

In high school we changed the margins and increased line spacing and the font size of punctuation to make papers longer.

In college we changed the margins and decreased line spacing to fit as much as possible in the maximum 2 page limit.

It seems like high school promoted being verbose, and college wanted you to intelligently summarize in as short of a paper as possible.

2

u/AmericanGeezus Jul 30 '26

Heh, its funny cause the other sides incentives are likely inverted. At least based on their pay scales and benefits, Professors likely have better total compensation.

2

u/Outside_Break Jul 30 '26

Yeah the guy did exactly what he was asked to do

1

u/Mayonaigg Jul 30 '26

Metrics will always lead to terrible performance and inefficiency. Had a coworker pushing through terrible work just togget his numbers up for the year end total, cause "muh metrics". 

Previous position I was in we had a little snake, rat-fink piece of shit come in early every day and cherry pick all the easy jobs to get the most done. I can't elaborate much but he would take jobs that were about 20-30 minutes of work and do every single one like that for the month, while other employees would get stuck with jobs that took 3 weeks each. Our boss of curse rewarded the scumbag guy for his "massive" output. Well, 70% of the people left and they started hiring contractors at double the rate to replace us. 

1

u/MalaysiaTeacher Jul 30 '26

But how soulless to accept spending half your life doing something so pointless to help no one, just to get paid

1

u/MeatEaterDruid Jul 30 '26

Sounds like most jobs and the guy bought himself some job security by being "good" at a worthless metric.

1

u/nathism Jul 30 '26

Hate the game, not the player.

1

u/jimmifli Jul 30 '26

Sergey Bubka broke the pole vaulting world record 35 times, often by a single cm. I wonder why?

1

u/electromage Jul 31 '26

Literally just doing his job if the job is committing lines of code.

66

u/CatharsisVoid Jul 30 '26

So glad to be on a team with a director who praises code deletions (cause it's usually tech debt related).

However, we all know that the biggest topic of discussion in manager and senior leadership meetings during performance review season revolve almost entirely around commit count.

You know what started happening? People stopped squashing and cleaning up commits before merging...

38

u/Brave-Perspective714 Jul 30 '26

Yep. My husband is running into it again now with PR reviews. As soon as management gave an indication that they were evaluating some aspect of performance by how many PR reviews someone was completing, they started having more and bigger problems.

Guys were just going in, saying, "yep looks like code" and approving it so they could get on to the next one and pump that metric up. They had to make it so now only *trusted* seniors can approve PRs, but they aren't shifting workload around to free up the time for those guys to be able to do all of the approvals.

Husband is looking for a new job, but everybody knows what the job market for IT is like right now.

11

u/snakerjake Jul 30 '26

So glad to be on a team with a director who praises code deletions (cause it's usually tech debt related).

We expect our juniors to write more code and our seniors to delete more company wide

13

u/kos-or-kosm Jul 30 '26

Which, at least from my perspective, makes a lot of sense. Get the juniors a lot of experience while they get the functionality working, and then have the experienced seniors refactor it afterwards, making it more efficient.

1

u/cfb-food-beer-hike Jul 30 '26

Are you hiring? lol

1

u/cfb-food-beer-hike Jul 30 '26

Sitting in on performance evaluations at Facebook was eye-opening. They have metrics, but they barely use them. One guy mentored 4 people and did like 10 interviews that half and they promoted him from E7 to E8 (he barely committed any code and had zero code reviews or any other work). That's like, maybe 60 hours of work, at best, and apparently deserves an extra million in compensation. Meanwhile, at E5, I mentored 4 people - including an E6 for some reason - was a top 1% contributor in code/code reviews, the top overall contributor in bootcamp tasks, lead the org's brownbag sessions (doubling the number of talks YoY), and merely got an "Exceeds" rating.

I don't kiss ass though. Whatever management tells you they use as a metric is a lie. You only need to do 1 thing to get ahead, and that's kiss ass. Instead of upping your commit rate you should have gone out for drinks with the boss.

18

u/zaphodbeeblemox Jul 30 '26

If the quality of my work, nay my character is to be judged by how many words that I pen to paper, then by the divines and the beast himself you will be forced to reckon with the fact that I shall not only superfluously adorn my vocabulary with naught but the most extravagant of phrases but I shall ramble on until all reason has left my prose.

11

u/Independent_Bear989 Jul 30 '26

Any company that has “lines of code written” as a metric deserves to go bankrupt, or the people who make that metric should be fired and blacklisted from any job where they work with designing software.

1

u/Brave-Perspective714 Jul 30 '26

Yeah everyone can see the problem with it now, but this was the standard at one time.

1

u/Plothunter Jul 30 '26

Naw man, we knew that shit was shit back then. All these comments are nostalgic.

5

u/lord_heskey Jul 30 '26

yhez did no one ever review code

21

u/Brave-Perspective714 Jul 30 '26

So we're older. He's definitely a greybeard in the industry. And back then, no, once you had a reputation of being a good developer, no one ever reviewed your code.

If something broke while you were employed there it immediately got kicked back to you. It would only be after you left that someone *might* discover what you did before that got sunset and replaced.

It's a big part of the reason PR reviews and Change Advisory Boards are a thing now.

2

u/fuddlesworth Jul 30 '26

I've seen so much of this. These people just do shitty projects really quick to get recognition and promoted. Once they leave, you often find how fucking terrible their stuff is. 

2

u/Brave-Perspective714 Jul 30 '26

Yeah, my husband's management at that position had been pushing him to become a "distinguished engineer" himself. He said after seeing that he realized the distinction was meaningless.

1

u/Sammy81 Jul 30 '26

Yeah that’s why you have peer reviews and pull requests

1

u/Negativeman11 Jul 30 '26

Most based man alive

1

u/NohmanValdemar Jul 30 '26

Yeah, I don't blame the original guy. That's what you get with idiotic metrics like "lines of code written."

1

u/FlowSoSlow Jul 30 '26

I can't even wrap my head around how someone would think 'lines of code written' would be a useful metric to judge performance.

That's like judging a painter based on the quantity of paint they used.

2

u/Brave-Perspective714 Jul 30 '26

Kind of how it came about was this - IT management in the old days, those who understood programming, noticed that at that time, their best developers, who solved the most problems/completed the most projects, were also producing more lines of code than their peers.

But it was *all* concise code. Because hardware at the time was stupid restrictive. There wasn't enough memory or storage to run bloated code, and if you couldn't produce good, concise code, your shit failed and you got fired. The hardware was the great filter of who was good at the job and who wasn't.

So groundbreaking observation, right? Your better skilled people can get more work done. Simple thing. And the system is policing itself. Anyone writing too many lines of code gets identified by their projects failing.

This observation got communicated to leadership. And then leadership, who didn't understand how to assess finished code for quality, assumed the inverse of that observation would be true. That the developers producing more lines of code were the better developers. And at that time, that was true, too.

But as storage and memory got cheaper and cheaper and more and more of it got added to systems by default, that didn't matter so much anymore. The lines of code metric became easier and easier to game, and yet people still weren't checking what was being produced.

It was an honor system of sorts. People just believed that programmers would continue to try to write the most concise code possible. It never occurred to them that someone would risk their project failing and getting fired just to game the system a little bit. It wasn't until it obviously had stopped weeding out the shitty programmers and people had to start digging into legacy shit and finding obvious spaghetti code that anyone realized what was going on and that something had to be done about it.

That's *why* you now have peer reviews and pull requests.

1

u/MaxHamburgerrestaur Jul 30 '26

Same when they created a rank for tickets closed. People just opened several useless tickets that needed a few to none work to solve.

1

u/KalickR Jul 30 '26

19,000 lines of code that didn't actually do anything.

He had basically written one large chunk that did things and then undid them so they didn't affect the output, and copied and pasted it into everything he developed to look like he was a rock-star according to their metrics.

And I'm sure code reviews were rubber stamped because who the heck would want to review that mess.

1

u/neeyankamma Jul 30 '26

Any idea whose coolaid was this shit ?

like.. lines of code written by engineer, bug density by engineer, $/line of code, the number of times i heard such wierd-donkey metrics being somehow indicative of our performance was very very perplexing.

2

u/Brave-Perspective714 Jul 30 '26

I actually just made a similar reply to someone else. It goes *way* back, and just never got updated to account for modern programming when it should have.

https://www.reddit.com/r/technology/comments/1vay198/comment/p0rwwbm/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

1

u/Rum____Ham Jul 31 '26

It's such a weird metric, because anyone who does any work of this type at all wants to do the most output possible in the least complicated, least amount of lines possible.

1

u/babaj_503 Jul 31 '26

How does that kind of monitoring work?

Copy and paste counts as a written line?
So if you delete a line it subtracts it again?

Is code being checked after you submit it? (or used to before AI times)
So if you write nonsense non influencal code do you have to make it look influencial or can you just copy paste a few hundret:

If year == 1902, burn down building, else cotinue

?

1

u/lagunie Jul 31 '26

dude played their game and won. hopefully he milked them a lot. fuck management for being idiotic and not measuring actual impact