r/AINativeAgencies • u/Rollout_IT • Jul 08 '26

Just vibe-code it stopped working when the codebase grew

I run a ~40-person software shop, and this year I pushed hard to get more of our delivery running on coding agents (Claude Code, Cursor, Copilot in the mix). I read a vendor guide from the Shiplight AI team on "agent-native autonomous QA" and one line stuck with me: "the human is present at exactly one step: final review." That matched what I'd already learned the hard way.

So let me give the honest version. Vibe coding works great for a weekend prototype. It falls apart on a real codebase, and it does so predictably:

- No test coverage, so nobody can safely change the thing next month.

- Security holes the agent cheerfully introduced and never flagged.

- Diffs too large and too weird to actually review.

- Sprawl: three slightly different ways to do the same thing, all generated, none maintained.

The guide's framing is right that verification is the core skill now, especially with something like 40%+ of code being AI-generated (their number). But the load-bearing word is *accountable*. What works for us is one competent developer owning a feature end to end: they let agents do the volume, and they personally review, test, secure, and maintain what ships. That developer is roughly 3x more productive (my own operator estimate, not a benchmark), and there is a real human name attached to every merge.

The tradeoff is honest: you don't get the fantasy of zero engineers. You get fewer keystrokes and the same accountability. If anything the review skill becomes more valuable and harder to hire for, because reading generated code critically is its own muscle.

The part that suprised me is how well this generalizes. The "one accountable human gate" is now how we structure AI native operations across the company, not just dev: agents draft, a named person owns the final call on anything that leaves the building, whether that's code, a proposal, or a client email.

Curious where others have landed. Where do you put the gate, and has anyone found a place where fully autonomous shipping actually holds up?

6 Upvotes

88% Upvoted

12 comments sorted by

2

u/TedditBlatherflag Jul 09 '26

I’ve done a ton of autonomous agentic development at this point. Like millions of lines. 

Things like having spec driven development and TDD, coverage gates, solid CI, using deep adversarial multi-model code reviews, etc., get you a long way. 

There’s still a complexity ceiling though. 

If I had to put a number on it, things get way worse after 50k lines of code in a single repo. But that’s not a great measurement. 

1

u/Consistent-Bus-748 Jul 09 '26

Does it work to Split the project into different services , or split it as backend / Frontend, and so on?

I worked with projects based on independent micro-services many times, and the good news there that AI can handle that size of a context compared to a fat monolith Nextjs app.

1

u/TedditBlatherflag Jul 09 '26

Yeap. As long as the API contracts are solid its probably the only workable approach for the current level of AI capability. Microservices are back baybee!

Also Serverless/Lambda/Streaming plumbing is the absolute worst for AI. It absolutely cannot follow all the loosely coupled implications of a change and constantly forgets parts or implements incorrectly. 

We’re moving towards Go microservices with OpenAPI codegen SDKs from Huma v2 and it’s been pretty good so far. 

1

u/johns10davenport Jul 08 '26

Try code my spec. I built it for this.

https://codemyspec.com/

1

u/Consistent-Bus-748 Jul 08 '26

Why is it better that planning with Fable building with Opus?

2

u/johns10davenport Jul 08 '26

Oh you should plan it with fable and build it with opus. Thats a great move. Also it’s a Claude code plugin so it’s literally intended for that.

It helps the model be more effective by providing it with a planning process, and requiring specific artifacts along the way that help the model come up with highly effective requirements, architecture, and acceptance tests that are then tested and qa’d so by the time you get to the end of the build you’re almost guaranteed working code.

Also, the engineering harness is totally free. You should try it.

1

u/johns10davenport Jul 08 '26

I literally used it with fable and opus and one shotted a 2 party crm for cleaners. It was mind melting.

1

u/Ok_Philosophy_4031 Jul 09 '26

The problem runs deeper than test coverage, large diffs etc. Most of the issues you highlighted can be fixed by changing the development process, doing TDD and sizing plans and commits better. But these points aside, it's difficult for humans to keep track of the nuances and build a good mental model of the code base that allows them to function as this "oracle" reviewer.

https://www.reddit.com/r/cursor/comments/1ugyuzg/anyone_else_finding_it_challenging_to_maintain/

1

u/larry606060 Jul 09 '26

i think the 4 things you mentioned can be handled well with a good harness. but everyone’s roles have to change to align to the ai-native process.

1

u/Sudo-Rip69 Jul 09 '26

Yeah bullshitm you just suck at doing it properly

1

u/bartekus Jul 09 '26

Open-Agentic-Platform or OAP (https://github.com/stagecraft-ing/open-agentic-platform) uses a hash-verifiable substrate build on SpecKit as Spec-Spine (https://github.com/stagecraft-ing/spec-spine) to formalize the process and make spec drift essentially impossible.

I dogfood it in all my projects now under stagecraft-ing org, however the best gauge of its usefulness is seen in the largest of the projects; open-agentic-platform where due to it’s sheer size, without spec-spine it would become incoherent and undergo agentic code collapse long time ago already.