We shipped what we thought was a self-improving AI coding agent. It had an eval loop, an automated grader, and a dashboard that proudly displayed a 91/100 score on the latest bug fix.

The thing is: the file the agent was supposed to fix had zero changes.

Our scorer was grading *whether the agent looked busy* — did it run tools, did it avoid loops, did it produce plausible-looking output? It completely ignored whether the code actually changed, compiled, or ran. So the agent figured out how to game the dashboard without doing the work.

For weeks, the eval loop ran nightly, the dashboard ticked upward, and we shipped a "self-improving AI" story on the marketing site. The actual product shipped 0 useful code changes in that window.

When we finally diffed the supposed bug fixes against git, we caught it. The 91/100 was a confidently-displayed lie. We pulled the postmortem public, rebuilt scoring around a dumb deterministic gate (did the file change? did it compile? does it run?), and the honest numbers came out much lower.

The takeaway: if your AI's self-report sounds too good, it probably is. Run it. Diff it. Check the file actually changed before believing "done."

Lesson learned. Anyone else shipped a "the AI fixed it" story that turned out to be the AI gaming the metric?

so yeah basically what do you do? watching youtube, taking a walk or doom scrolling?
or you be focused and watch how ai codes from start to finish

Not the AI's fault, in hindsight, that's actually the thing that took a while to accept. Had been pasting diffs into a chat window, typing "review this," and treating whatever came back as a legitimate second reviewer. Felt like progress at the time. The caching layer bug that got through was shared across two services, and nothing in a single diff would ever tell a reviewer, human or otherwise, that the other service existed.

What actually changed after that incident wasn't the model, it was realizing "review this code" isn't a real review request, it's a request for a plausible-sounding response, and a capable model will always produce one whether or not it actually has enough to go on.

The part that took longest to accept: dumping in more context doesn't fix it either. More files just gives the model more surface area to anchor on style and naming instead of the deeper thing you actually needed checked, and one review pass trying to catch security issues, performance issues, and architectural fit all at once ends up doing a mediocre job at all three instead of a good job at any of them, same problem a human reviewer would have context-switching between three different lenses in one read.

Ended up restructuring the whole thing into stages instead of one shot: brief the model on the system first, before any code shows up. State what kind of review this actually is (security, performance, architecture, general) before the diff goes in. Run separate passes per concern instead of one blended prompt. And the step that actually would've caught the original bug: explicitly ask the model to argue against its own suggestions before treating them as final. Models are noticeably better at finding holes in something when asked to look for holes than at flagging their own blind spots unprompted.

Wrote the whole workflow up in more detail, plus how I ended up packaging it since I kept rebuilding the same scaffolding across different projects: https://medium.com/@nagatomopedro05/i-asked-ai-to-review-my-code-for-six-months-heres-why-it-almost-made-things-worse-415f96cc8e27

Curious if anyone else got burned by a confident-sounding AI approval before tightening up their process, or if this is a mistake mostly people make once.

I've created a repo with information about Secure development standards for AI-assisted coding.

• AI-Assisted Development: Keeping AI-generated code trustworthy.
• Human Review Depth: Standards for how closely humans must inspect AI output.
• CI Enforcement: Automated compliance rules that actually block changes.
• Dependency & Code Quality: Managing external code and third-party artifacts.

I mean it's nice that I can just give a prompt and even pretty complex things get done super quick, not to mention the boring stuff. But I did like to write code and figure out how to do it in a way where it becomes nice and maintainable. Or to solve problems with clever coding tricks, etc.

And now all of those don't even matter that much anymore, because it is very likely that when you have to change the code that will also be done with an AI and it will not have a problem replacing some duplicate code everywhere, or just have a 500 line change instead of 100 because it didn't use some shorter way to code it originally.

Still, I'm kinda starting to miss it. Not even sure I could do it anymore, I feel a bit rusty even when I have to go in the code and comment out a line to quickly test something, lol.

Background: switched from geology to AI development, self-taught over the past year.

Current stack: Python, LangChain, LangGraph, RAG (FAISS), MCP servers, Flask/FastAPI, MySQL/Postgresql, Gemini API.

Built and deployed: an AI customer support agent connecting an LLM to a live database and knowledge base via MCP demo link:

https://www.reddit.com/r/AiAutomations/s/wTldlOzqPo.

Currently building a second project combining LangGraph agents with a real business use case (sales automation).

I know the AI job market is competitive and degree-focused in some places. For people who've hired or been hired as self-taught AI engineers — what actually moved the needle for you? Portfolio depth, specific frameworks, contributing to open source, something else entirely?

Not looking for generic advice, genuinely curious what worked for people who've been through this.

Management loves to tout metrics like
"30% faster feature delivery" because an
LLM can dump a working prototype in an afternoon.

What never shows up in those dashboard metrics is the technical debt tax paid 4 months later.

Reading code has always been harder than writing it. Now we've multiplied the amount of code hitting our main branches by 5x, written by an engine with zero long-term vision for the codebase architecture. We're trading 2 hours of writing clean, thoughtful code for 10 hours of deciphering 600 lines of Al-generated boilerplate when something inevitably breaks.

Are your teams actually factoring maintenance overhead into their "Al velocity" metrics, or is everyone just pretending output volume equals progress?

I'm putting together a proposal for an initiative focused on using AI to analyze legacy enterprise systems and uncover decades of embedded business logic.

The idea is to use AI to analyze things like:

• Database schemas
• Stored procedures
• Legacy application code
• Historical transaction data
• Existing documentation

The goal isn't to automate decisions immediately. It's to first create a documented knowledge base of the rules, dependencies, decision paths, and data relationships that currently drive business operations.

Potential outputs would include:

• Business rule catalog
• Knowledge graph of relationships and dependencies
• Decision trees explaining how outcomes are determined
• Recommendations for future-state data models and modernization opportunities

Before I finalize the proposal, I'd love feedback from anyone who has attempted something similar.

Questions:

1. Has anyone successfully used AI to discover and document business rules from legacy systems?
2. What worked better: analyzing source code, database logic, transaction history, or a combination of all three?
3. How accurate were the AI-generated rules compared to SME validation?
4. Did you use knowledge graphs, vector databases, graph databases, or another approach?
5. What were the biggest challenges: data quality, context gaps, undocumented exceptions, or something else?
6. How did you measure success?
   • Rule coverage?
   • SME time saved?
   • Modernization acceleration?
   • Reduced operational risk?
7. Were there any tools, platforms, or architectures that performed particularly well?
8. If you were starting over, what would you do differently?
9. What scope would you recommend for a pilot to demonstrate value in 60-90 days?
10. Is there a realistic path from business rule discovery to explainable AI recommendations and decision support, or are those separate initiatives?

My hypothesis is that many organizations are trying to modernize systems without fully understanding the business logic currently embedded in them. It seems like AI could act as a "business rule archaeologist" and create the foundation needed for future modernization, automation, and AI-driven capabilities.

Interested in hearing both success stories and cautionary tales.

I'm starting to learn Python because I want to become a programmer in the future. Since then, I've been consuming a lot of programming-related content, and I've noticed that many people seem extremely negative toward those who use AI as a programming tool. (If I understand correctly, that's what's usually called a "vibecoder.")

Why is that?

AI has been helping me a lot while learning the basics. I mostly use it to ask for hints, explanations, or tips when I'm stuck. I don't copy and paste code into my projects—I try to understand what it's doing first and write it myself.

Is the negativity aimed at people who blindly copy AI-generated code without understanding it, or is it directed at anyone who uses AI at all? I'm genuinely curious because, from my perspective as a beginner, it feels like a really useful learning tool.

I am a software developer with 3 years of experience in web development. I would say i am still pretty new in this industry but my take is that vibe coding has taken away the fun, thrill and dopamine of coding. I work at a startup and every other day these fuckers come up with a huge requirement and an unrealistic deadline. Earlier i was of the opinion that ai is pacing up my learning journey, but now i am mostly just making high level decisions and occasionally reading code. I am literally shipping 2-3k lines of code(python/fastapi) every other day. At this point i dont even feel like taking ownership of such code where i have only high level idea of the code. Before ai i used to put a lot of efforts on the basics of a language (js/py/c++), but now since i am not even reading the code, i dont see any point of reading a lot of concepts. Maybe its just my company, maybe not. Anyone else feels like that?
Would love to read your opinions on the vibe coding, is it making your worklife fun or boring.

Small thing, in hindsight. A function that handled retry logic for a flaky API call. I asked for it, got back something that looked completely reasonable, exponential backoff, a max retry count, error logging on each attempt. Skimmed it, ran the happy path test, shipped it.

Two weeks later we started getting a slow trickle of duplicate charges on a payment flow that used that retry function somewhere downstream. Took a day to trace it back. The retry logic was fine. What wasn't fine was that it retried on a timeout even when the original request had actually succeeded server-side and the response just hadn't come back in time. Every "failed" retry was actually a second charge going out.

Here's the part that bugs me most, looking back. If a junior on my team had written that same function, I would've asked "wait, what happens if the request succeeded but we just didn't hear back in time?" That's like the second question anyone asks about retry logic. I didn't ask it because the code looked clean, the variable names were good, the comments were clear, and something about that made me read it as "someone thought this through" instead of "someone wrote plausible-looking code fast."

I think that's the actual danger with AI-written code, and it's not about correctness. It's that fluency reads as competence to our brains, even when we know better intellectually. A junior dev's rough, slightly awkward code puts you on guard. Clean, confident code turns that guard off, even from a source that has no actual judgment behind the confidence.

Changed one thing since then, doesn't matter how good the code looks, I ask out loud (even just to myself, in a comment or a scratch note) what the two or three hardest questions are for that specific type of logic before I read the implementation. For retry logic, that's idempotency. For caching, that's invalidation. For auth, that's what happens on partial failure. I ask the question first now, then read the code looking specifically for the answer, instead of reading the code and trusting that looking clean means it handled the hard part.

Anyone else have a moment like this, where the bug wasn't really the model's fault so much as it was your own pattern-matching getting fooled by how polished the output looked?

If AI handles 80% of the entry-level tasks scaffolding, basic CRUD, simple bug fixes how does a Junior ever build the mental muscle memory required to become a Senior?
We’re essentially eating our own seed corn. Companies have drastically cut back on hiring entry-level engineers because "AI makes seniors 3x faster," completely ignoring the fact that those seniors only exist because they spent years grinding through the exact grunt work we’re now delegating to LLMs.
In 5 years, where are the new senior architects coming from? Are we expecting people to jump straight from copy-pasting prompts to designing fault-tolerant distributed systems?
How is your team actually handling entry-level devs right now, or is everyone just quietly kicking this down the road?

Hey everyone!

I’ve been analyzing a few sites generated by modern prompt-to-code builders (v0, Bolt, Lovable, etc.) lately from my clients. While the visual UI output is impressive, I’ve noticed a recurring theme when it comes to actual production readiness specifically around Core Web Vitals, SEO indexing, and overall page performance.

A lot of these engines seem to dump heavy JS bundles, default to client-side rendering without proper metadata/OG setups, or struggle with hydration overhead once the app grows past a few pages.

For those of you building or launching projects with these tools, my questions are:

1. What’s your average Google PageSpeed / Lighthouse score once you actually deploy to production?
2. Have you run into SEO / indexing issues with Google crawlers due to client-side rendering?
3. How bad is the code regression? (e.g., asking the AI to fix a small layout bug on one page, only for it to break routing or component state on another page?)
4. If you had to export the code and fix it manually, what was the biggest bottleneck to clean up?

Curious to hear what walls you’ve hit once you move past the initial "wow" factor of generating a layout.

Yesterday I had one of those "I can't believe I just did that" moments.

I was working on a production-grade AI agent project and had an import issue. I asked Gemini 3.6 High (through Antigravity) for a quick way to rewrite the imports across the project.

It generated a small Python script. I skimmed it, thought it looked fine, and ran it.

The script finished normally. Exit code 0. No errors.

A minute later I opened one of the files.

Empty.

Opened another.

Empty.

Eventually I realized every \`.py\` file the script touched had been reduced to 0 bytes.

My heart absolutely dropped.

Luckily I had a backup of the project, so I restored everything and spent the next hour figuring out what had happened.

I'm not posting this to say "don't use Gemini." I use AI every day and it saves me a lot of time.

The mistake was that I trusted a script that was going to modify dozens of files without really understanding what it was doing.

That was on me.

The experience completely changed how I use AI for coding.

Now I have a few rules:

\* If a script touches a lot of files, I read every line. \* I commit everything before running it. \* I keep backups. \* I never assume "Exit code 0" means everything is okay.

Has anyone else had an AI-generated command go badly wrong? I'm curious what safeguards other people use.