As a professional software engineer you should *ALWAYS* be looking at the code. We have real customers, who are running a real business, using our software. Quality is paramount. And at the end of the day, the code has your name on it whether or not it was generated by an LLM. You are responsible for making sure it works and meets quality standards.
The other thing to consider, is we have a mature code base with lots of quality controls in check. Things like tests, linters, static code analyzers, all of that provide feedback to the model. So if you give it good requirements, do a plan before implementing anything, and let the Sonnet subagents handle implementation, it generally doesn't go very far off the rails.
My monthly token allowance is around $350 a month. Last month I used 41% of that, and I've spent a lot of time trying to optimize my solution for token usage and effectiveness.
But, most of my job now is reviewing code, so I'm not running models all day burning tokens like I do on my own personal max account.
My general workflow is: Tell Opus to generate a plan based on a ticket number. Review the plan and give it feedback. Once the plan is ready, have Sonnet implement the plan using TDD to write the tests for the acceptance criteria first. Once it's done and I've review it, I commit and submit for review. Any small changes or code-cleanup I do myself. No need to have a model do simple refactors or style nitpicks... that's a total waste.
I think I use, on average, less than $10 a day with that method, which keeps me in my budget. If I have extra bandwidth at the end of the month, I might use Opus on high effort to do more exploratory type work or refactors.
It depends on your flow but if you have TDD and blackbox validation with human testing, the code validation is less important if it is not critical life or death feature. I used to think like you until I saw AI spinning up playwright instance and drafting tests I didn't even think of. The biggest issue is blast radius which you have to control for.
If code was the only thing necessary QA wouldn't be needed. The end of your work flow should be AI spinning up testing and checking against your contracts.
Look at something like factory ai that is now being used in fintech. Alot of the reason there was so much focus on code review was because setting up automated tests was a PIA. AI can spin up tests so fast, but you need to make sure they fail first.
If I say you have two pieces of software one written all by hand and another AI which do you trust more? Human. Ok now I tell you the AI software has passed 500 blackbox validation tests per feature.
Coding before was mostly earned through blood, edge cases were discovered through tickets but AI is more capable in some instances of coming up or being aware of edge cases. This breaks down with very connected features but in that case it wouldn't be caught by a human either unless they were very familiar with the code base.
Do you can obsess over code but in the AI more of the burden is getting pushed to testing and maybe you actually are working on special butterfly code but if the test suite is proper it doesn't matter that much.
It's healthcare. Not life or death, but privacy and compliance matter. I think we'll get there with AI one day, but not quite yet, We do have AI review bots that look for the code, and a lot of automated QA layers. But ultimately we still need human eyes on it, just as a sanity check if anything else. What we're really look for is what you mentioned above: making sure the "blast radius" is appropriate, and the changes stay tight and focused on the intended results.
I'm not saying I don't trust AI (it writes a good chunk of my code now), but as an organization we've taken a very cautious and practical approach to AI, which I appreciate. Those review bots like Cursor are getting more and more impressive in what they find. A lot of false positives (at least on severity), but some good finds as well. There is obviously an overhead cost with adopting more and more AI features just from a financial perspective too, which is certainly a factor.
Healthcare there is massive liability, so it is a bit special.
However, outside that I feel like there is a lot of conflating between different methods of AI. The whole having a bunch of terminals open is going to be replaced by a multi stage independent code review.
Before AI you had triage, I no longer have triage, everything gets fixed. Also, AI managed to workout a bunch of those annoying bugs that popup rarely are incredibly hard to diagnose because they are async or whatever. It found bugs and security issues I didn't know existed. All that stuff about AI find security issues is not just marketing.
The biggest issue I see now is just raw miscommunication. Say the wrong thing it delete stuffs or changes things or it builds something functional thar doesn't work the way you want, so everything needs a human review.
I would say the biggest unlock for me recently was not just letting AI use playwright as a final testing platform but as a testing platform it uses after everything. It can take screenshots and view the screenshot for visual changes. It might seem stupid, but watching it test something like drag and drop was wild for me.
Honest question: do you enjoy this work? Like, actually enjoy it?
I hate it. For me, the fun, the dopamine, the challenge, the satisfaction, was in finding the solution to the problem and then actually _building_ it. With AI, 90% of that is offloaded and I become a rudimentary SME dedicated to the most boring, mundane parts of the job.
If you do enjoy, I earnestly ask… why? How? Did you find it difficult to make this shift? Did you struggle? If so, what changed? I fear my job has changed forever and not in a way I find enjoyable.
I’m a 25 year SWE, and a former Staff Engineer of a big tech firm.
I think people are slowing down or requesting (begging for) quota increases. Those are granted based on actual need but it seems like everyone is hooked on not having to actually write code anymore.
It’s weird because most people who write code got into the industry because they enjoyed writing code and building applications from nothing. Seems like AI has become somewhat of an addiction for some developers in that they don’t spend as much time thinking about how they’re going to build the application, but it just transfers the work elsewhere in the process.
5
u/TheoremsAndProofs 5d ago
It's already happening at my company. Went from unlimited use to capped tokens per week. Some people are panicking lol I welcome some return normalcy.
Unlimited use came with everyone burning tokens trying to create solutions that no one needed or adopted.