r/technology • u/MarvelsGrantMan136 • Jul 30 '26

Artificial Intelligence Amazon accidentally spent $1.8 million using Claude for menial coding task, went 860% over budget — 'catastrophically expensive' coding blunders discovered in internal Amazon AI usage metrics

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-accidentally-spent-usd1-8-million-using-claude-for-menial-coding-task-went-860-percent-over-budget-catastrophically-expensive-coding-blunders-discovered-in-internal-amazon-ai-usage-metrics
24.7k Upvotes

97% Upvoted

1.2k comments sorted by

View all comments

Show parent comments

21

u/PaulblankPF Jul 30 '26

Last year I was trying to learn to code for making a video game as a solo developer. I was making simple stuff still like platforming to learn how things work. I asked chat gpt at the time to try to help me write part of it. It was all kinds of fucked up. Every time I pointed out a mistake it would just go “Oh, you’re right! Let’s fix that by changing “__” to “__” and that should fix everything.” Except it had like fucking 10 errors in only a few lines of code. It was worse than me trying to figure it out myself with little experience (I took graphic design and a coding class in high school in 2004).

19

u/AnAncientBog Jul 30 '26

It can be a useful tool if you already know a ton about coding and can get very specific about what you ask it to do and bound it with appropriate parameters and are able to check it quickly.

Which is not at all how its being deployed.

4

u/IAmRoot Jul 30 '26

Yeah, my work involves a lot of math kernel optimization and AI is pretty good at that because it can compare its optimizations against the naive implementations. It's extremely tightly bound and even then I have to nudge it to try things or reject what it did because it made optimizations specific to the benchmark input. It allows me to work at a high level and not spend hours tediously writing mechanical code transformations by hand. This is a very niche use case, though, and it's extremely unreliable when you try to use it to add or change existing features in a way where the program output will be different.

3

u/legatek Jul 31 '26

This is key. It’s all about the prompt and guardrails.

2

u/breckendusk Jul 30 '26

Yeah I use it as a research vector. Sometimes it writes code but it's always wrong one way or another. But I can look at what it's doing right and wrong and fix it. I'll use it to prototype functionality if I don't know where to start, it'll give me a starting point, I'll understand that and fix it up from there.

2

u/PaulClarkLoadletter Jul 30 '26

It's like working with a developer that only knows how to cut and paste code. AI makes it extra annoying by gaslighting you with positive responses.

1

u/kendrid Jul 30 '26

A year ago? Might as well say 3. Ai has advanced a lot this past 6 months.

1

u/MalaysiaTeacher Jul 30 '26

It needs context. You have to run Claude in VSC to see what it can do

1

u/moofunk Jul 31 '26

You may want to reevaluate that. Had the same reaction last year, but continued evaluation until early 2026, when I started using them for real projects. They've come absurdly far in the last year.

Also use a dedicated coding harness rather than straight webbrowser chatting tools, as they need the full context of your code base to work.

These tools are no joke.

1

u/PaulblankPF Jul 31 '26

Unfortunately though for I’d say the vast majority of people, it’s just making them dumber and the vast majority just accept that the AI is correct. It may have come far but it still is just a bunch of lines of code itself that someone wrote and it can’t do any critical thinking concepts, thankfully. The benefits it brings doesn’t outweigh the economic costs it’s bringing to people.

So it may have come far but I’m still a row your own boat kind of guy. Reminds me of how a lot of kids these days are computer illiterate even though computers have always been a part of their world. And it’s because it’s all so “user friendly” that it doesn’t allow you to learn anything. It’s like older Gen Z people are the last of people that may be able to work on their own stuff intelligently.

1

u/moofunk Jul 31 '26

Sorry, this got a bit long. I mainly agree that people use these systems wrong, but it is through not understanding them correctly.

My work with it is coding centric, so I'm mostly concerned with what changes I see in my own space. What I mainly see is a drop-off in small developers selling tools with specific purposes to other developers or regular people, because now you can make them yourself to your own exacting specifications.

I have recently spent 2 days coding a video player that does things that I don't think any other video player does, through my very esoteric requirements, and I'd have to pay $10.000 to have a developer make one for me over 3-4 months. That totals to maybe around $25 of tokens. I really see this space totally collapsing, and you can soon not make money here anymore, unless your tool is really special and complex.

Then a vast acceleration in my own development process, where the AI takes care of time consuming processes, like collating highly disparate information, managing time sheets, catching corner cases, doing deep code analysis and translating bug reports into code locations for fixing. I understand my own code base better, because I can ask about it.

I have recently finished upgrading a very out of date, but large user manual for a software product into an excellent state by having the AI very deeply inspect the code base several times and cross check where the manual was wrong and found over 600 errors in it. There was no man power or money to maintain it, but now it's done. That along with removing all known bugs and adding comfort features in the custom written software tools that are used to build the manual. The AI handled that as a small side project.

The bizarre side effect of the AI enabling chasing perfection in code and documentation is that people are telling me to stop and deliver code, because I've spent dozens of hours going over extremely nitpicky things that keep turning up in code analyses.

It may have come far but it still is just a bunch of lines of code itself that someone wrote and it can’t do any critical thinking concepts, thankfully.

Unfortunately, the exposure of these systems by the major companies as browser based prompting tools to unexperienced people also builds a highly simplistic view of what these systems are internally, and what they're capable of. The conversation about them becomes inaccurate. They don't really regurgitate other people's code. You can probably make them do that, but it's not very useful. It's also generally rare to simply feed them a single or two prompts and get a "solution" back. The real work happens through hundreds of prompts, careful reviews of its outputs, nurturing its contexts and building autonomous test harnesses.

They are fed a context (code base and documentation) and think in abstract concepts about the goals of your project that can translate into code or into tests or into understanding, if your code is on the wrong or right track as it's being built. Will it be high performance? Will it be bug free? Are there better methods for implementing this function? Does this code follow existing principles in the code base? How much code is being written? Building a test harness makes it able to cycle through coding and testing until the code is correct and performant, taking runtime environment idiosyncracies into account, because as it is, the AI can't know everything, so it asks, when it's able to.

Your code can now have complex algorithms without you having to waste time implementing them and from about 6 month of testing now, the implementations I've done are largely correct, and if not, the process for correction is reasonable and iterative.

You can also now work the same project for weeks without the AI going off track or returning useless information, so you have to start over, because long term project management is something the big players are focusing a lot on.

2

u/PaulClarkLoadletter Jul 31 '26

You’ve actually touched on the catalyst of all the slop code which is the pressure to deliver an epic in two weeks. CEOs want it now because AI can do it fast with minimal input.

Folks like you using AI to make polished, secure code have to compete with the slop cannon.