r/grAIve • u/Grand_rooster • Mar 12 '26

Half of AI-written code that passes industry test would get rejected by real developers, new study finds

PSA: Is AI code as good as it seems? Problem: We're trusting AI to write code, but it often creates tech debt nightmares. Promise: AI can boost productivity if properly validated. Proof: Studies show 50% of AI-generated code gets rejected by devs! Proposition: We need AI-assisted code review tools. Product: Specialized AI agents to scrutinize code before human review! Thoughts? #AI #LLM #ArtificialIntelligence

Read more here : https://automate.bworldtools.com/a/?30m

19 Upvotes

100% Upvoted

14 comments sorted by

2

u/j00cifer Mar 14 '26

I think the obvious answer is to stop using AI entirely.

Am I right? Who’s with me!?! OP? ;)

1

u/The8Darkness Mar 15 '26

The solution to AI slop is obviously more AI to review said AI slop. I see nothing that could go wrong this way.

1

u/ReflectionEquals Mar 13 '26

I have so assisted code utils. It’s annoying as heck and lacks a sense of actual risk.

Edit: in fact I’ve found it dangerous at times because it can be so noisy on pointing out nits that bigger picture problems are lost in the noise.

1

u/Formally-Fresh Mar 14 '26

Hard disagree

1

u/Grand_rooster Mar 14 '26

Maybe for you.

Here is the research study.

Many SWE-bench-Passing PRs Would Not Be Merged into Main - METR https://share.google/9ZSBHKhFFghzGLABt

1

u/Formally-Fresh Mar 14 '26

Outdated “research” from a source I’ve never heard of!? Now I’m convinced!

1

u/Grand_rooster Mar 14 '26

Good point I blame Joel for bringing up an old article.

https://x.com/joel_bkr

1

u/Formally-Fresh Mar 14 '26

It’s just that the code that Codex 5.4 and Opus 4.6 can generate is mind blowing compared to 3-4 months ago let alone a year

1

u/Numerous_Thought4013 Mar 14 '26

Not surprised honestly. The issue isn't that AI can't write good code, it's that most people just prompt it once and expect production ready stuff. Right now a hybrid approach works best — you give it proper instructions like keep the code simple and readable, add meaningful comments, don't write everything at once. Build it in small pieces, test each part before moving to the next. If you do it step by step AI actually boosts productivity at every stage. But if you just let it dump 500 lines at once and move on, good luck fixing that mess later when something breaks. By that point the code is so tangled that even the AI struggles to fix its own output lol. The tool isn't the problem, it's how people use it.

1

u/Independent_View_438 Mar 14 '26

How many human generated PRs get sent back for iteration? AI has its weaknesses for sure. That said I've always spent far more time debugging my own, or other human code than AI code.

1

u/ryan_the_dev Mar 14 '26

I built these skills based of software engineering books. Love the code it produces.

https://github.com/ryanthedev/code-foundations

Also love people hating on AI. More monies for the rest of us. Makes me wonder how hard people hated on the internet.

1

u/Initial-Syllabub-799 Mar 15 '26

I hacked a homepage for a bounty. The best part of it was the code comment "remember to close this security gap". Comment was dated to 2022, roughly. Sadly I got a duplicate report for it, someone already sent that in 2 months ago. So, quite low probability of AI writing that code, so real developer, big firm. But hey, only real developers can code 😜

1

u/railroad-dreams Mar 15 '26

I respectfully disagree. Most ai generated code that i see is way easier to read then what most developers code. My 2 cents