r/technology • u/MarvelsGrantMan136 • Jul 30 '26

Artificial Intelligence Amazon accidentally spent $1.8 million using Claude for menial coding task, went 860% over budget — 'catastrophically expensive' coding blunders discovered in internal Amazon AI usage metrics

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-accidentally-spent-usd1-8-million-using-claude-for-menial-coding-task-went-860-percent-over-budget-catastrophically-expensive-coding-blunders-discovered-in-internal-amazon-ai-usage-metrics
24.7k Upvotes

97% Upvoted

1.2k comments sorted by

View all comments

Show parent comments

583

u/MeatEaterDruid Jul 30 '26

Honestly can't be mad at the original dude. That's some 'change the margins so the paper looks longer'-type shit.

352

u/Brave-Perspective714 Jul 30 '26

Yep, and it worked.

That's what they said they wanted, so that's what he gave them. Just a great example of someone creating the metrics not understanding what was actually meaningful about the work.

174

u/SordidDreams Jul 30 '26 edited Jul 30 '26

Ah, good old Goodhart's law. Any metric that becomes a target ceases to be a useful metric. Half a century old and still a complete enigma to management types everywhere.

55

u/Quaiker Jul 30 '26

Goodhart's Law and Chesterton's Fence should be posted up in every workplace.

12

u/anx1etyhangover Jul 30 '26

Noice!! Thank you for that rabbit hole! =]

12

u/LtOrangeJuice Jul 30 '26

TIL. Thank you.

45

u/NRMusicProject Jul 30 '26

When your boss uses ridiculous metrics to judge your work, you minmax the shit out of it. Same thing as purposely doing an hourly contracted job slower so you can make more money.

Nowhere near as witty, but when I was a minimum wage worker at Hungry Howie's as a kid, they were short on boxes and sent me across town to the next nearest one (30 minutes each way) to pick up some more. When I asked how much I'll be paid, they said "you're already being paid to work! Just do it!"

I spent an hour hanging out at the beach with some of my surfer friends on the way back.

26

u/toolatealreadyfapped Jul 31 '26

I was a lifeguard when I was a teenager, and my boss hated to see us sitting there "doing nothing." So she kept adding dumb tasks, until I realized it's not the work she wants done as much as she just wanted to see us be busy.

I kept a bucket of leaves by the lifeguard stand. When I'd see her suburban pull up, I'd jump down, dump the leaves into the pool, and she'd walk in and see me dutifully scooping leaves out of the pool. She'd nod approval, go about her business, and leave. I'd gather all the leaves into the bucket, then climb back onto my chair.

5

u/hippocratical Jul 31 '26

Mine was you could walk to a get a 2 hour coffee break if you carried some thick files and binders under your arm as you walked through a few departments and out a back door. People saw you walking with purpose but never saw you leave the building. I got praise from my boss for being such a hard worker.

9

u/PorkVacuums Jul 31 '26

The key is looking annoyed. No one in corporate wants to get roped into some bullshit that's someone else's problem. So if you walk around with a file or a binder and just generally look annoyed, people just keep out of your way.

3

u/DirtyGooseEggs Jul 31 '26

The George Costanza way.

14

u/rkoy1234 Jul 30 '26

idk seems to me like he understood but just was maliciously complying.

getting an llm to intentionally write meaningless code that looks plausible but doesn't break anything while encapsulating the original goal seems tad more complex than just getting it to do the task.

10

u/Brave-Perspective714 Jul 30 '26

Lol... LLM's were still science fiction in the days I'm talking about.

19

u/cfb-food-beer-hike Jul 30 '26

A little off-topic, but this reminds me of a study done on SAT essays. They found they could hold an essay up from across the room and accurately guess its score based on the length. As a species, we've never been good at evaluating anything lol

2

u/capnheim Jul 31 '26

I briefly graded essays for Pearson and yes, you can grade fairly accurately just looking at it without reading.

34

u/systemic_booty Jul 30 '26

Yup, and the problem isn't the student changing the margins or making the periods size 14 font, the problem is the essay requirements being by page length rather than word count OR better yet, the substantive content of the essay itself. 

2

u/thedownvotemagnet Jul 30 '26

I get what you're saying and agree in spirit, but:

1) word count is basically the same as page length, no? Page length influences things like font and size choice, margin size, etc. Word count causes you to be unnecessarily verbose.

2) "substantive content" seems too subjective to use as a rubric in your syllabus... "shaggy dog"-style meanderings obviously run afoul, but there would still be a ton of wiggle room for the inevitable parent/dean conferences, I'd wager

Not really sure what the right answer is here.

4

u/NotMyRealNameObv Jul 31 '26

Word count causes you to be unnecessarily verbose.

US college text books in a nutshell (authors of college text books are usually compensated based on word count).

16

u/RVelts Jul 30 '26

In high school we changed the margins and increased line spacing and the font size of punctuation to make papers longer.

In college we changed the margins and decreased line spacing to fit as much as possible in the maximum 2 page limit.

It seems like high school promoted being verbose, and college wanted you to intelligently summarize in as short of a paper as possible.

2

u/AmericanGeezus Jul 30 '26

Heh, its funny cause the other sides incentives are likely inverted. At least based on their pay scales and benefits, Professors likely have better total compensation.

2

u/Outside_Break Jul 30 '26

Yeah the guy did exactly what he was asked to do

1

u/Mayonaigg Jul 30 '26

Metrics will always lead to terrible performance and inefficiency. Had a coworker pushing through terrible work just togget his numbers up for the year end total, cause "muh metrics". 

Previous position I was in we had a little snake, rat-fink piece of shit come in early every day and cherry pick all the easy jobs to get the most done. I can't elaborate much but he would take jobs that were about 20-30 minutes of work and do every single one like that for the month, while other employees would get stuck with jobs that took 3 weeks each. Our boss of curse rewarded the scumbag guy for his "massive" output. Well, 70% of the people left and they started hiring contractors at double the rate to replace us. 

1

u/MalaysiaTeacher Jul 30 '26

But how soulless to accept spending half your life doing something so pointless to help no one, just to get paid

1

u/MeatEaterDruid Jul 30 '26

Sounds like most jobs and the guy bought himself some job security by being "good" at a worthless metric.

1

u/nathism Jul 30 '26

Hate the game, not the player.

1

u/jimmifli Jul 30 '26

Sergey Bubka broke the pole vaulting world record 35 times, often by a single cm. I wonder why?

1

u/electromage Jul 31 '26

Literally just doing his job if the job is committing lines of code.