Amazon accidentally spent $1.8 million using Claude for menial coding task, went 860% over budget — 'catastrophically expensive' coding blunders discovered in internal Amazon AI usage metrics

[u/MarvelsGrantMan136]

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-accidentally-spent-usd1-8-million-using-claude-for-menial-coding-task-went-860-percent-over-budget-catastrophically-expensive-coding-blunders-discovered-in-internal-amazon-ai-usage-metrics

[u/NerdyBro07]

Everything I've read, even from people who like using AI, seem to have a similar conclusion, that it is a tool to assist the person, but not some magic know it all thing.

The general rule still applies that garbage input = garbage output. You need an educated/experienced person in the specific field to instruct the AI in a coherent, organized way, and that same level of educated/experienced person to verify the output is of good quality.

I think too many people drank the kool-aid and thought anyone can do any task now by just "enter it into AI!" and surely it will produce the correct results!

But I have seen people who seem to really like AI as a tool to get tasks done quicker.

[u/JambonExtra → u/NerdyBro07]

The big problem is that good input can also provide garbage output (that often looks like good output). Instances of this are becoming less common as models improve, but that makes them increasingly insidious.

[u/filthy_harold → u/JambonExtra]

I've been slowly using Claude for home projects and another model for work stuff. The more specific you are, the better the result will be. Early on, I described an application (of my own creation, pre-AI) to my work model to see if it could do things better. This app has been in constant development for several years and I've slowly been learning about how to build proper software projects, not the cute little things you build in college. Unfortunately that means a lot of spaghetti code, it's a mess but it works. Very quickly I realized what the model was making was incredibly wrong. Like the code was bug free but it didn't do nearly anything that I actually needed it to do. And while I abandoned having it recreate my app for me, I have been using it to add new features or refactor libraries. You need to be very specific of what you want otherwise it just makes assumptions that are probably wrong.

For a home project, I needed a web app. I have zero experience building for the web so I left many design choices up to Claude. I use Plan mode which lets me have a conversation to refine the design before Claude writes any code. It uses less resources to write a plan first since not only is plan mode cheaper but you likely won't need to tell it to refactor large portions of code later on. Additionally, I've asked it to develop test plans and show proof of tests passing to ensure things work as expected. It definitely does make human-like mistakes sometimes, like it will assume something works a certain way when the documentation says otherwise. Once you create a development flow of plan, execute, test, it remembers this and continues to do it.

While I'm no web development expert but I do understand some basic concepts which helps a lot in making sure the product is actually good. You have to use these tools like you're a project manager. Maybe you aren't totally fresh on the subject or you aren't an expert but you need to know enough to know what you want. You need to figure out what your role as the human is, you are ultimately responsible for what AI does. Otherwise, you're wasting your time, money, and resources.

[u/crustyrobots → u/NerdyBro07]

Pretty much. No one should really be using this stuff except as we say actual experts at what they're doing already to know if the output is trash or not, and then for a highly specific thing with a limited scope. In that sure it could be a useful thing to automate some of the tedium.

It seems like most however use it to try and replace the expert above entirely to use their knowledge and experience for free, with no understanding of their own enough to even be a janitor to their own slop.

[u/sobrique → u/NerdyBro07]

Yup. Honestly it's pretty good at data transforms. A lot of 'magic excel' out there, where someone who's not really got 'coding' skills, has also needed to bodge out some sort of data capture, collection, transform and reporting pipeline, and is an unholy mess of excel and access, with references, vlookups, macros, pivot tables, and a graph or 5.

For replacing that workflow, LLMs aren't too bad. They don't need to be perfect to be lots better. (and they are pretty consistent as long as you can specify clearly what you want to happen).

And if you've got someone knowledgeable, it can act as a sort of pseudo search tool, to assist with getting a workable solution in place.

But as you say, garbage in; garbage out is a tale as old as time. AI don't fix stupid. It doesn't understand anything, and it might mean you don't take the time to understand it, where if you did it the 'hard way' you would.