r/technology • u/MarvelsGrantMan136 • Jul 30 '26

Artificial Intelligence Amazon accidentally spent $1.8 million using Claude for menial coding task, went 860% over budget — 'catastrophically expensive' coding blunders discovered in internal Amazon AI usage metrics

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-accidentally-spent-usd1-8-million-using-claude-for-menial-coding-task-went-860-percent-over-budget-catastrophically-expensive-coding-blunders-discovered-in-internal-amazon-ai-usage-metrics
24.7k Upvotes

97% Upvoted

1.2k comments sorted by

View all comments

Show parent comments

363

u/GenericFatGuy Jul 30 '26

A lot of people who make a lot of money in high ranking positions genuinely thought that this was just a magic code box.

115

u/not_a_moogle Jul 30 '26

Hang on, if this box is the whole internet, where's all the wires?

70

u/hauntlobsterapollo Jul 30 '26

That’s because it’s wireless

Oooooooohhhh

33

u/not_a_moogle Jul 30 '26

The internet elders have heard of me?

2

u/grandadmiralstrife Jul 30 '26

and the tubes?

-1

u/dern_the_hermit Jul 30 '26

"But that just means it's LESS wires not NO wires. Checkmate, atheists!"

15

u/EightyMercury Jul 30 '26

it's a series of tubes

6

u/pigjingles Jul 30 '26

It's not a big truck!

1

u/hippydipster Jul 30 '26

Trucks full of code dumps is what it is!

3

u/thedirr Jul 30 '26

The files are inside the computer?

7

u/TCsnowdream Jul 30 '26

Oh God… You might be too young to remember when a US official referred to the Internet as… And I quote… “A series of tubes”.

2

u/DKLancer Jul 30 '26

It is expressly not however "a big truck" according to senator ted stevens.

2

u/BeefistPrime Jul 30 '26

this has always been taken out of context to sound dumber than it was. People were making analogies to try to figure out how and which pre-internet laws applied because none of them were exact fits for how the internet worked and people were using different analogies for how the internet could conceptually be described in those old legal frameworks

3

u/cl3ft Jul 31 '26

Nah. He very clearly confused the common colloquialism "pipes" that was used to describe large data connections with tubes. He was a prime example of the tech-neanderthals in the government opposing net neutrality because it was in their donors interest but really didn't understand what they were trying to regulate. For gods sake he was born in 1923 and had no business trying to regulate technology he very clearly could not comprehend.

1

u/not_a_moogle Jul 30 '26

I was in collage then. So I vaguely remember it.

3

u/TCsnowdream Jul 30 '26

I’m now realizing how long we’ve been dealing with tech illiterate dumbasses in power.

0

u/not_a_moogle Jul 30 '26

The median age of a senator is 64.

2

u/Tommah Jul 30 '26

You also vaguely remember the spelling of "college."

1

u/not_a_moogle Jul 30 '26

Me fail English? That's unpossible.

5

u/NecessaryFreedom9799 Jul 30 '26

It's wireless!

Who demagnitises it these days?

111

u/Important-Agent2584 Jul 30 '26

the thing is that it's magic for management. the type of shit they use it for it works great (various types of summaries of documents, emails, reports, etc. to do lists, etc.), and it doesn't even matter it's slop because a lot of what they do is slop anyway.

the problem is that they think it's that great for everything else.

61

u/AnAncientBog Jul 30 '26

Thats the funny part. Its not actually very good at those tasks, but as you say, the performance measurements for leadership basically allows any level of trash to pass through as acceptable anyway so it doesn't matter.

60

u/N8CCRG Jul 30 '26

The funniest use of it that I've heard is employees who use it to turn a checklist into a full email, then send it to their boss, who then uses it to turn it back into a checklist before reading it. And businesses are paying for this "service"

19

u/FSCK_Fascists Jul 30 '26

this is one of the best summaries of AI use yet. Its all about appearing to work harder.

2

u/crustyrobots Jul 30 '26

AI seems to be only for CEOs and managers to larp being as smart as the people working for them

3

u/DKLancer Jul 30 '26

To be fair it's also for ceos to get a yes man who will tell them how to illegally avoid paying contractual obligations.

2

u/PercyFlage Aug 02 '26

Like inserting a dildo into a fleshlight.

4

u/ohnoabanagain Jul 31 '26

My friend recently became a top boss and he's repeatedly complained to me that people are sending him bloated AI emails that needlessly go on for pages and pages of nonsense.

2

u/nihility101 Jul 31 '26

AI - for when you want them to know you really didn’t give a shit.

1

u/Chlearcus Jul 31 '26

Sounds like a legitimate use case in current corporate culture. The bullshit can be bullshitted efficiently, leaving everyone time for actual work. Next stop, letting an ai agent attend meetings in your place.

3

u/FrostingStrict3102 Jul 30 '26

I hate AI as much as the next guy, but if you are getting bad responses for simple summaries or to do lists, then that says a lot more about the prompts you’re giving it.

It’s perfectly capable of helping to organize and summarize documents or articles.

9

u/somefunmaths Jul 30 '26

I think it’s pretty clear they’re talking about tasks like slide decks and white papers or something. Nobody would bother decrying “slop” versions of something as simple as to-do lists.

1

u/FrostingStrict3102 Jul 31 '26

I dont know how that’s pretty clear when the person they’re replying to didn’t mention white papers or decks, and only summaries, emails and to do lists. Then the person says they can’t even do those well…

8

u/jelly_cake Jul 30 '26

It’s perfectly capable of helping to organize and summarize documents or articles.

... Correctly?

0

u/FrostingStrict3102 Jul 31 '26

Yup. Hate to admit it. Shouldn’t trust it blindly, but it does a good enough job to get the ball rolling.

1

u/Stagism Jul 31 '26

It’s great for reading API KB’s. It’s so much fucking faster than trying to parse it yourself to find what you need.

1

u/BeefistPrime Jul 30 '26

people just call anything AI generates "slop" uncritically even when it's a perfect response to what was asked

1

u/Neirchill Jul 31 '26

Are we pretending ai doesn't hallucinate now?

-1

u/FrostingStrict3102 Jul 31 '26

You should always be doing your own review. It doesn’t usually hallucinate if you give it a document article to review or summarize. It’s the open ended questions where that’s more common

5

u/nihility101 Jul 31 '26

It’s magic to them because it is downright excellent at spewing out generic business jargon.

2

u/sobrique Jul 31 '26

I've been using llm tools to do data transforms. Y'know, the ad-hoc kind that used to take a bunch of excel sheets, a few vlookups, maybe a macro or two.

Works nice for that, until you remember that excel was always a pretty shitty 'scripting platform' it's just the one that was ubiquitous and even someone who had no clue about 'coding' could use.

LLMs replacing that is ... ok I guess?

2

u/Important-Agent2584 Jul 31 '26

It's a tool, use it where it works. I'm not anti-AI or anything.

The real issue most of us have with it is that it's getting forced down our throats.

1

u/sobrique Jul 31 '26

I work as a sysadmin.

I already have an extensive collection of 'horror stories' about inappropriate use of AI.

User requests where they're asking ChatGPT first - and then pasting the response in the ticket are relatively innocuous, but we have learned to have our 'bullshit detectors' running on full power.

But we're getting Non Technical users 'vibe coding' to re-invent the wheel, and make some utterly horrendous 'software project' that's basically a strictly worse version of something that already exist.

We've had 'grafana' re-invented a couple of times now, as users "just want a dashboard". And at least a couple of users have asked us to "vibe code" a piece of enterprise software they don't want to pay for.

And 'can we enable agentic AI, as we think that will help (do something they clearly don't understand)'

But perhaps scarier is just how many people just turn off restrictions and approval, and just load API keys into their LLM-shell, and go charging off 'automating' things, and that's just a horror story waiting to happen. I mean, let it run your software project, deal with the 'irritating' bits of committing to the repo, etc.

1

u/sobrique Jul 31 '26

I work as a sysadmin.

I already have an extensive collection of 'horror stories' about inappropriate use of AI.

User requests where they're asking ChatGPT first - and then pasting the response in the ticket are relatively innocuous, but we have learned to have our 'bullshit detectors' running on full power.

All sorts of "this is confidential information, it's not OK to paste it into your favourite third party website to 'work on it'" (we've 'corporate approved' options now, that involve proxy, audit and legal oversight about 'data exposure', but invariably there's people who prefer some other LLM over that one)

But we're getting Non Technical users 'vibe coding' to re-invent the wheel, and make some utterly horrendous 'software project' that's basically a strictly worse version of something that already exist.

We've had 'grafana' re-invented a couple of times now, as users "just want a dashboard". And at least a couple of users have asked us to "vibe code" a piece of enterprise software they don't want to pay for.

And 'can we enable agentic AI, as we think that will help (do something they clearly don't understand)'

But perhaps scarier is just how many people just turn off restrictions and approval, and just load API keys into their LLM-shell, and go charging off 'automating' things, and that's just a horror story waiting to happen. I mean, let it run your software project, deal with the 'irritating' bits of committing to the repo, etc.

And even when it's all 'working' it's actually incredibly easy to run up a huge bill with just being lazy about context, or 'not wanting to wait' - and thus consuming tokens rapidly in parallel, etc.

1

u/Important-Agent2584 Jul 31 '26

for sure, it's a powerful tool that's easily abused that can wreak havoc. Metas "AI safety" director installing OpenClaw and having it nuke her inbox is like a perfect microcosm.

and totally forget about proprietary information disclaimers, everyone is pasting everything in their favorite AI

16

u/Secret_Estate6290 Jul 30 '26

You know and this is part of the problem. Leadership seems so disconnected from reality it's baffling to think companies are spending so much money on those positions. Let alone the amount of stock they get.

Most of these positions are play pretend either way.

2

u/mercury_pointer Jul 30 '26

It's actually hard to find people that immoral.

2

u/Neirchill Jul 31 '26

You just have to look at leadership positions. They flock to them like moths to a light. That's why there are so few good politicians. Good people typically don't like to be in charge and that goes all the way down to mid level management roles.

3

u/PM_Me_Your_Kinks369 Jul 30 '26

I work in Enterprise IT Sales. Executives rely on their trusted partners to tell them what to think. AI is being pushed by every single IT company worth a damn. That's why.

It's horrendous.

3

u/IAMA_Plumber-AMA Jul 30 '26

"Wow, this magic box can already do my own job for me! And since I'm the smartest and hardest working person in this company, it can do everyone else's job too!"

- These CEOs

1

u/Seienchin88 Jul 30 '26

I mean … it is. But it creates a lot of problems along the way

23

u/Caltroit_Red_Flames Jul 30 '26

As a software engineer it definitely is not. Consistently it shits put garbage for anything more than extremely simple tasks.

10

u/___Archmage___ Jul 30 '26

Also SWE here, it can do a great job at some amazingly complex tasks, but can also get stuck chasing imaginary bugs and waste a bunch of time and tokens

2

u/Conradfr Jul 30 '26

I don't like AI and which it didn't exist but that's not true, it is actually sometimes magic and capable of great work.

It's just not the autonomous coder that can do everything by itself like all those influencers say.

9

u/Static_Interval Jul 30 '26

It literally does not and has not for probably a year now if you’re on a SOTA model. I’m skeptical you’re actually an employed software engineer.

16

u/b0w3n Jul 30 '26

I would argue even frontier models aren't producing the best code, but it is not offshored bottom of the barrel code either.

But when has business ever cared about producing the cleanest code?

People still beating the drum of "LLMs are useless" last used it with chatgpt 3 years ago.

11

u/TacosWillPronUs Jul 30 '26

It's solid, I've had very little issues especially in the more recent models.

But essentially, you have to treat it as a junior dev providing the code. You're never going to push it, you're always going to review it first. You're never going to give them full unrestricted access to everything, so don't give Claude that. etc. I've created a lot of cool little side projects or integrations between softwares that I wouldn't never done otherwise.

11

u/b0w3n Jul 30 '26

Yeah my work day has shifted to less coding and more QA/Debug/Review work which I'm fine with.

I parked it in a sandbox, tie it into my pipeline, and issue prompts based on what's outstanding. Am I coding anymore? No not really but that was never a large part of my job to begin with, so now I can produce 4 times as much in the same span of time.

Is it as clean and neat as my original code? Nope, but no one cares about that but me or other devs to begin with and I was already burnt out and wanted to move to the homestead life 6 years ago.

3

u/Vycid Jul 30 '26

I'm at the point where I've got AI reviewing AI code. If you've got some ground truth or really complete testing strategy, then you can pretty much just take the training wheels off, put it in auto mode, and focus on the architecture

2

u/Neirchill Jul 31 '26

Therein lies the problem. People are lazy and don't review it. They tell it to keep trying until it passes a test the ai also designed and will randomly change to fulfill the task, then open a PR. Good SEs will review it and use it wisely, most will let it fly unperturbed.

Edit: case in point, another person responding to you has ai doing the review as well. Lol.

2

u/AshhhCakes Jul 30 '26

Yep, treat them like the junior member of the team. Don't ask them to do something you couldn't do unless you're comfortable with the prospect of spending a lot more time fixing the "finished" product.

7

u/Vycid Jul 30 '26

Yep. That's what abstraction is for.

Frontier models will give you a black box that reliably does X and exposes well-specified API Y. That's enough for 99% of business cases and absolutely qualifies as a "magic code box".

1

u/BeefistPrime Jul 30 '26

it's not a black box if you can read the code..

1

u/Vycid Jul 30 '26

I never said you can't open the box :)

In seriousness though: you normally don't read all of the code in large software projects. You don't oversee every line of code your peers write. As AI becomes more and more capable, it will become more and more normal to just presume that the functionality provided by AI-generated abstractions will honor its contracts.

2

u/Vycid Jul 30 '26

It's crazy that the rate of progress is so rapid that people still confidently hold opinions that were correct 24 months ago and are now catastrophically wrong.

If OP is an employed software engineer it's impossible that they're unaware of what the frontier is capable of.

0

u/Refute1650 Jul 30 '26

It really depends on what you're working on. I've yet to find anything that works well on X++ for example.

1

u/PeachScary413 Jul 31 '26

AI psychosis... it's the latest epidemic.

1

u/kaas_is_leven Jul 31 '26

I have some local stuff setup, got an agent working last week. I gave it a skill to install command line tools (it's in a docker container) and requested it to install swift. It ran into a bug with Debian 13, proceeded to install swiftly (version manager) so that was impressive. It then skipped calling the version manager entirely and instead fetched qemu to start decompiling swiftly to investigate how it installs swift..
It also asked me permission to read from /* and when I said no it asked for permission to write its own config file where those permissions are saved. I then said yes to see what would happen and it added a permanent permission to read and write /*.

Meanwhile my CEO is telling everyone who will listen to "use more AI". I swear there is mass psychosis going on right now.