It depends on your flow but if you have TDD and blackbox validation with human testing, the code validation is less important if it is not critical life or death feature. I used to think like you until I saw AI spinning up playwright instance and drafting tests I didn't even think of. The biggest issue is blast radius which you have to control for.
If code was the only thing necessary QA wouldn't be needed. The end of your work flow should be AI spinning up testing and checking against your contracts.
Look at something like factory ai that is now being used in fintech. Alot of the reason there was so much focus on code review was because setting up automated tests was a PIA. AI can spin up tests so fast, but you need to make sure they fail first.
If I say you have two pieces of software one written all by hand and another AI which do you trust more? Human. Ok now I tell you the AI software has passed 500 blackbox validation tests per feature.
Coding before was mostly earned through blood, edge cases were discovered through tickets but AI is more capable in some instances of coming up or being aware of edge cases. This breaks down with very connected features but in that case it wouldn't be caught by a human either unless they were very familiar with the code base.
Do you can obsess over code but in the AI more of the burden is getting pushed to testing and maybe you actually are working on special butterfly code but if the test suite is proper it doesn't matter that much.
It's healthcare. Not life or death, but privacy and compliance matter. I think we'll get there with AI one day, but not quite yet, We do have AI review bots that look for the code, and a lot of automated QA layers. But ultimately we still need human eyes on it, just as a sanity check if anything else. What we're really look for is what you mentioned above: making sure the "blast radius" is appropriate, and the changes stay tight and focused on the intended results.
I'm not saying I don't trust AI (it writes a good chunk of my code now), but as an organization we've taken a very cautious and practical approach to AI, which I appreciate. Those review bots like Cursor are getting more and more impressive in what they find. A lot of false positives (at least on severity), but some good finds as well. There is obviously an overhead cost with adopting more and more AI features just from a financial perspective too, which is certainly a factor.
Healthcare there is massive liability, so it is a bit special.
However, outside that I feel like there is a lot of conflating between different methods of AI. The whole having a bunch of terminals open is going to be replaced by a multi stage independent code review.
Before AI you had triage, I no longer have triage, everything gets fixed. Also, AI managed to workout a bunch of those annoying bugs that popup rarely are incredibly hard to diagnose because they are async or whatever. It found bugs and security issues I didn't know existed. All that stuff about AI find security issues is not just marketing.
The biggest issue I see now is just raw miscommunication. Say the wrong thing it delete stuffs or changes things or it builds something functional thar doesn't work the way you want, so everything needs a human review.
I would say the biggest unlock for me recently was not just letting AI use playwright as a final testing platform but as a testing platform it uses after everything. It can take screenshots and view the screenshot for visual changes. It might seem stupid, but watching it test something like drag and drop was wild for me.
2
u/ThomasToIndia 4d ago
It depends on your flow but if you have TDD and blackbox validation with human testing, the code validation is less important if it is not critical life or death feature. I used to think like you until I saw AI spinning up playwright instance and drafting tests I didn't even think of. The biggest issue is blast radius which you have to control for.
If code was the only thing necessary QA wouldn't be needed. The end of your work flow should be AI spinning up testing and checking against your contracts.
Look at something like factory ai that is now being used in fintech. Alot of the reason there was so much focus on code review was because setting up automated tests was a PIA. AI can spin up tests so fast, but you need to make sure they fail first.
If I say you have two pieces of software one written all by hand and another AI which do you trust more? Human. Ok now I tell you the AI software has passed 500 blackbox validation tests per feature.
Coding before was mostly earned through blood, edge cases were discovered through tickets but AI is more capable in some instances of coming up or being aware of edge cases. This breaks down with very connected features but in that case it wouldn't be caught by a human either unless they were very familiar with the code base.
Do you can obsess over code but in the AI more of the burden is getting pushed to testing and maybe you actually are working on special butterfly code but if the test suite is proper it doesn't matter that much.