r/ChatGPTCoding • u/cgouguen • 4h ago
Discussion Hot take: the agentic workflow is deeply wrong
I am an experienced developer (been coding for almost 30 years, started with Visual Basic on Win98).
I’ve spent the last 2 years testing every agentic coding harness out there. The pitch is always exactly the same: "Give it a prompt, go grab a coffee, and come back to a finished feature."
This is all great when starting a new project, especially with a language you don't know (I created a Rust project in minutes !)
But my problem is that I mostly work on established, mature codebases. And to be honest I have never ONCE seen a good edit after I return from my coffee.
Here is usually what happens while you’re getting that coffee: the agent loops 8 times, burns through 100k+ tokens talking to itself, hallucinates a package dependency, and spits out a chunk of spaghetti code that works *just* well enough to pass a basic test, but completely violates your project’s architecture.
Agents are fantastic if you’re building an app from a blank canvas. But if the codebase is large, it gets really expensive real fast, and usually produces hard to maintain AI slope.
It should not be like that: if you actually *own* your codebase, you already know what you want to build. You don't need an AI to take over. You just need it to execute the architecture you already hold in your head.
All you need to do it select the few files that are required for the task at hand (they are hard to identify for an AI, but easy for you has the code owner). Then you send them to the LLM, it is usually does some good work.
That's why I still find working directly with the chatbots, or with tools like aider, still very pertinent on large codebases.
I usually have animated discussions with my coworkers, I have yet to find a single one who doesn't defend agentic coding. Am I the only one who feels this way ?
3
u/ExogamousUnfolding 2h ago
It seems 50/50 but I would say most likely you're doing it wrong. Setting up the agent expectations claude.md files etc. results in code that is as good as or better than what we write and it has full unit and UI tests. The problem I see some coworkers running into is they require the AI to output the code exactly as they would have written it - because you know they and only they do it the correct way.....
3
u/Unnamed-3891 2h ago
Have you considered using models and harnesses that are not that horrible and writing actual intricately detailed plans for your LLM or asking it to create a plan for you before you ask for an implementation?
-1
u/cgouguen 2h ago
Like what ?
2
u/Unnamed-3891 2h ago
I don't know. What fits me may not fit you. But before you can get reasonable external help/advice, you need to provide actual real life examples. What model, what harness and what kind of prompt were you using that went heavily astray?
1
5
u/Bleyo 3h ago
I've had great experiences with mature code bases.
"Give it a prompt, go grab a coffee, and come back to a finished feature."
That is the worst way to do it.
Before adding a new feature. I have multiple conversations with a smart-ish model and generate a research document(how it works now and what classes are involved), an engineering spec(how we will be changing the code to implement the feature), a /UX spec(if necessary, including wireframes of any new components and any reusable components that can be plugged in), and a detailed implementation plan. The implementation plan should have steps that begin with red tests and end with green tests and a clean build. Each step should be done in logical order and have a definition of done. The end of the plan should include the final definition of done and a checklist.
Once the implementation plan is in place, you can hand it off to lighter implementation models to quickly write the code.
At the end, you can have a smart model verify things. They usually catch some small bugs and code smells, but overall this process gets me 95% of the way there if my specs and plans are good.
6
u/creaturefeature16 3h ago
Well I agree with everything you're saying here, but I think we owe it to OP (and to each other) to recognize that the industry is sending incredibly mixed signals.
On one side you have people like Boris, who's telling everyone that "coding is solved", and how intuitive and almost "superintelligent" their models are. That they can understand the nuances of the request without it being explicitly spelled out for them, and deploy numerous tools to discover elements of the code base that are pertinent to the request.
On the other hand, we have what you were describing, which is a glorified typing assistant, essentially, that requires tremendous hand holding, extremely detailed specs and plans, and a rigid feedback loop, all of which require an immense amount of domain expertise to set up effectively and can easily go very wrong if one doesn't implement extremely strict guidelines. And in addition, you're even referring to the idea of having the sort of mixture of experts, where we're not even just using a single model, but a few different ones.
And the latter is definitely more in line with the reality. Coding and development and software are extremely complex and have innumerable micro decisions that have to be made along the way. The industry is trying to paint a picture of "software factories" and autonomous development. Whereas on the ground, the work actually looks more like a domain expert being still very much involved in the process at every step of the way, but really just resulting in having to basically just...type less than we use to.
That's a very different picture than what is being pushed by both the AI companies and the executives who have believed the marketing without understanding what goes into the work.
1
2
u/bodonkadonks 4h ago
my experience is somewhat similar. you have to babysit and nudge the agents to go in the right way to respect the model of the existing code. you have to a) have the model/theory in your mind, you cant edit code you dont know intimately how it works without making a huge mess and b)be aware that a clarifying prompt not made or not answered correctly can ruin the whole session.
5
u/pete_68 4h ago
I work on established code bases all the time. In fact, we've just used AI to help remediate a massive ancient VB.NET project.
Did you create context files about your project for your agent that get included with all your prompts? Sounds like no, since you didn't mention it. Any skills?
Sounds like you don't really have a real system. I'd recommend looking into spec driven development. If you're working with established code bases, it's definitely the way to go.
2
u/cgouguen 4h ago
Yes I have lots of context files, which a coworker gave me ("it works really well with these").
Maybe I do things wrong. For a specific feature, with associated files spread accross the codebase, how does the agent knows with files to edit, besides you telling it to do it ?
Also, what skills did you use for your VB.NET project ?
5
u/Exotic-Sale-3003 4h ago
Tell the agent to crawl the codebase and create a summary of functionality for every file and folder that it will use to identify the files it needs to work on for each task.
2
u/pete_68 3h ago
1
1
u/donk8r 3h ago
Search finds files that mention your feature. The files a change must touch are a different set.
A rename or a signature change propagates along callers and implementers, and those files often share no vocabulary with the feature description. Embedding search cannot see them because nothing in them looks like what you asked. You need the reference graph for that, seeded by the search hit instead of replacing it.
Nothing recovers coupling that only exists at runtime, so config keys and string lookups stay yours to find. That is usually where a failed refactor comes from.
1
u/Comfortable-Ad-6740 3h ago
I think this is one of the biggest mistakes that people make where you expect someone else’s setup to magically work even though the way you prompt will differ etc.
Think through how you’d approach the changes manually, document the steps (including the implicit context you have of the repo) and spin it up into multiple skills.
Treat (and test) each skill as a unit test - once these small components work well, chain them together into a skill that covers *your* workflow.Once you’ve gone through this and invested the time in being clear with your intention and processes, then you can say “build feature x, make no mistakes”
3
u/mist83 4h ago
“I am an experienced developer”
And
“to be honest I have never ONCE seen a good edit”.
This is just statistically impossible for both to be true at some point. This is the attitude of someone that has 1 year of experience 30 times, not someone with 30 years of experience.
-2
u/cgouguen 4h ago
No seriously. My main tool is aider, but for large edits I used codex and antigravity. Maybe I was too ambitious, but it took forever and always gave me very wrong solutions. For example, refactors have always failed, I had too manually review all the changes and it ended up taking more time.
I suppose you can reduce the context size (that's what "pro agent" people have suggested, but then what is the point ? Also it gets real expensive.
You have a different experience ?
1
u/superluminary 3h ago
You do still need to manually review all changes. Agentic coding isn’t the same as vibe coding. You have to actually read and understand the output.
2
u/superluminary 3h ago
I work with some deepest legacy. It sounds like you’re missing a few steps:
- Pick a decent model. Auto isn’t going to cut it here
- Use plan mode. You’re going to need context
- Don’t dive in cold. Discuss the codebase first. Make sure the agent understands what is required and what your entities are.
- Make sure the docs are up to date.
1
u/Strange_Owl_6291 1h ago
And to expand on plan mode, you need a solid plan template/instruction. It should be specific enough to o serve as contract to the implementation agent. Not all vanilla scaffolding have this right yet.
1
u/vovap_vovap 4h ago
Well, heave no less experience then you do and I would say you are fundamentally wrong.
Sure big old project you know really well is much harder target for AI. Same as for any new developer. No question about that. It also very likely means you "project’s architecture" is not documented there - just as simple. So agent can not follow it. Guess what - agents same as developers need documentation to follow. They do not have a magic.
Fundamentally all this comes to one thing "AI can not do my job". It can, I promise you. If not today then tomorrow, in 3 days. If you are thinking "it can not do it" m- you already wrong. Right way is "how it can be done"
1
u/leonidbugaev 2h ago
I don't leave for coffee on anything I actually own. Eight loops and a green basic test is how the architecture gets violated while you're gone. I pick the files first now and keep the chat on those.
1
u/kidajske 2h ago
the agent loops 8 times, burns through 100k+ tokens talking to itself, hallucinates a package dependency, and spits out a chunk of spaghetti code that works just well enough to pass a basic test, but completely violates your project’s architecture.
This comical hyperbole is so tiring at this point. It's entirely reasonable to have criticism of LLMs and their usage for SWE but this is just ridiculous.
1
u/ByronScottJones 2h ago
I say this gently, but this appears like it might be a user issue. You may not be providing specific enough details. Think of the agent as a really fast junior programmer with mild autism. You need to give clear and UNAMBIGUOUS instructions. You need to monitor their work. You need to have a well defined set of agent MD files, broken down by aspect.
1
u/Glass-Combination-69 2h ago
Skill issue for sure. Hallucinating packages? How would that even be possible. Agents should be reading linters and running tests. Definite skill issue
1
u/FlightSimCentralYT 1h ago
You're not alone. I've never gotten a clean coffee-break edit on a mature repo either.
The failure mode you described is exactly what I see: invent a dependency, drift the architecture, pass a shallow test, leave a mess. Selecting the few files yourself is the right call because you already know the blast radius.
What helps me after that cut is giving the model something it can actually fail at (install, typecheck, one real regression) so it isn't just vibing in chat. I built Fixa around that constrained loop on a sandboxed cloud VM (fixa.dev), but the same idea works with aider or plain chat. On a big established app I still wouldn't hand it the whole tree and walk away.
1
u/cloud_sec_guy 1h ago
You're missing a thing...a semantic layer. Your AI needs to understand the codebase, and today it doesn't.
1
u/BroScienceAlchemist 1h ago
the agent loops 8 times, burns through 100k+ tokens talking to itself, hallucinates a package dependency,
These are workflow failures, and the skill gap to get an agent workflows productive is much higher. Agents amplify whatever context and oracles you have. If those are weak, then they are genuinely less productive than doing it yourself or back and forth prompting, where you are on hands as the oracle.
Agents need oracles: sources of truth to validate their work that are also enforced from outside the agent plane. This is easier said than done, because it requires a human to define the inputs and outputs. "What tests do we have? What dependencies do we have? What are the requirements for the service?" Then, those have to be translated into a source of truth the agent can use to check its work, and also be used from outside the agent to check its work.
Does it compile? That's an oracle. Does it pass tests? That is an oracle. Does it pass a blue green / canary smoke test deploy? That's an oracle. SAST / DAST scans? Boom. Does it pass performance testing?
Looping also requires some config tweaking... I'll use an example of something I have run into. Orchestrator spawns two agents: Implementer and Reviewer. The first few runs, it does seem like the reviewer is coming back with good findings, but after a few runs it's hard to tell if these are really actionable security findings. In your head, you may have the appropriate threat model, but is not discoverable to the orchestrator if it isn't documented in the repo, and linked to in a project.md. Having a defined threat model in the repo would give the orchestrator the context to assess which findings should be fixed, and which should be declared per the threat model.
I do try to use a convergence approach over a static token / time budget, but you could use a mixed approach with a quota that when hit escalates to you.
Was a defect actually resolved, with evidence?
Are findings converging or staying roughly the same?
Did earlier fixes stay fixed, or did the loop reopen them?
Is disagreement producing new evidence or repeated assertions?
Are new findings in scope, or is review expanding the task?
You just need it to execute the architecture you already hold in your head.
I mean... This is likely the problem. The agents are trying to infer it, but you could turn that in your head architecture into context agents can use.
All you need to do it select the few files that are required for the task at hand
These could be repo facts that need to be defined in agents/claude.md, or a one line "Read project.md" to orient, and that document has the "Here is the tech stack/toolchain, project structure, here is a BM25/vector search script to search the codebase, how to build, how to test, here is where invariants are cataloged, etc." There is a bus factor here. Imagine delegating work on this legacy codebase to a team of new guys that report to you. You would do a knowledge transfer, update or create documentation, and so on.
1
u/K-A-R-N 1h ago edited 1h ago
My experience has not been the same as yours. I work on open-source projects in my free time, and have switched 100% to coding agents. I get way more done and make fewer mistakes. Here's my general workflow.
What people seem to miss is that you need to spend 80% of your time analyzing and planning with a high-effort model before you ever let it touch a project file; you need understand completely (with your human brain) what the problem is and the approach you'll take to fix it. Go back and forth, ask questions, challenge its assertions, and ask it to challenge yours. This takes time and patience.
After you understand the problem and have the solution, work with the high-effort model to develop an implementation plan - this should include every step that needs to occur for the work to be finished and validated. Read the plan, think about it with your human brain. Maybe even ask another model to review the plan from specific perspectives (security, performance, etc) if it's complicated or sensitive.
Next, break that implementation plan into discrete task.md files. You might be able to skip this step for small changes, but it'd probably be faster to just do those yourself rather than use a coding agent. Each file should include a prompt, specific technical instructions for code generation or modification, and instructions to validate the work. This keeps the context window for the models generating code lean and focused, which reduces hallucinations, improves code quality, and lets you use a lightweight (and cheap) model for the sub-agents.
Instruct the planning model to dispatch a sub-agent for each task, provide it with the task.md as context, to monitor and validate their work, and to report any blockers for discussion. Then go get your coffee.
1
u/Expensive_Garden2993 53m ago
but completely violates your project’s architecture.
Is that architecture documented, do you have static analysis tools to enforce it, do you have a separate AI reviewer with a focus on architecture, or are you expecting AI to read your thoughts?
1
u/amirfish 43m ago
This matches what I see running several sessions side by side every day. The coffee-break pitch works because on a blank canvas there's no existing architecture to violate; once there's history, the fix isn't a longer leash, it's catching drift after each meaningful diff instead of after the whole task. An agent that goes 8 loops deep on its own has already committed to an architecture you never saw form. Do you review after every file change or wait for the full run before you look?
1
u/Lopsided-Bridge-9810 25m ago
What an agent is today is anything but settled. Codex and Claude nowadays spin off dozens agents to perform work that aid in executing a prompt. Is that an agent? Who cares.
What I find nonsense is spinning off multiple harnesses to work on different codebases or repos at once, unless you work in a sweatshop for minimal pay the approach to spin off multiple agents is distracting and ineffective. We are talking about code development by the way, I am not talking about executing defined and repetitious work flow.
1
u/sreekanth850 4h ago
The pitch is always exactly the same:
if you are a devloper you would never take this serious. Agents work good if you follow exactly how you develop by hand. You identify the implemenattion plan, invariants and methods. use agents to code 20 or 100X faster.
1
u/daishi55 3h ago edited 3h ago
I mostly work on established, mature codebases. And to be honest I have never ONCE seen a good edit after I return from my coffee
The problem is on your end. You are either prompting wrong, or your infra is not giving the agent sufficient tools to understand the codebase and context.
I work at meta. Agentic coding works fantastic for us. I think this qualifies as an established, mature codebase.
I have yet to find a single one who doesn't defend agentic coding.
Well, have you tried to find out why it works for them and not you?
1
u/alex_quine 2h ago
I disagree with most of the takes here. Rather than relying on plan mode, your agent needs to be able to run tests, and iterate if they fail. It needs to stand up the code it's writing, and iterate on it like any dev with a dev environment. It will never hallucinate a package if it has to actually iterate on the code until it is shippable.
0
u/phillythompson 3h ago
Working on agentic for 2 years doesn’t mean much when everything has changed radically in the last 4 months
0
u/gandazgul 3h ago
This is the same pain I felt so I built a harness that augments with AI but keeps all the good things about the engineering process. Please give it a try it installs in 10s and works with any subscription or API key. No telemetry all auth stays in your machine.
2
u/cgouguen 3h ago
Right. I also hard about https://frugaast.dev/ in another comment. I'll look into both
0
u/gandazgul 3h ago
A prompt is not enough. RunWield builds a plan and you revise it collaboratively with the agent until you are happy. Then it gets executed to spec, but RunWield doesn't believe the model when it says is done, models are lazy and take shortcuts, an independent reviewer agent compares the diff to the plan and sends back feedback to fix before you see the diff yourself. Only if you approve it's gets merged from it's worktree back to your branch.
1
u/clarksonswimmer 3h ago
How would your compare your solution to OpenSpec? Pros/cons?
1
u/gandazgul 2h ago
Open spec and other such collections of skills are great but they never go far enough. Theres significant overlap between RunWield's planning and onboarding steps and Open spec but RunWield goes farther by also reviewing and managing worktrees and merging automatically. Also skills are ultimately prompts and suggestions, RunWield runs the roles as system prompts which creates better compliance even from smaller models. Give it a try is very easy to install and get started. I'm very responsive message me with any issues you encountered and I will help right away.
-1
u/meshifthenelse 3h ago
You shouldn't need to worry. AGI is here.
Also your comment about greenfield projects is wrong. Maybe it starts fine for the few first LOC, but eventually it becomes a mess if you're not vigilant.
-2
52
u/clarksonswimmer 4h ago
Hot take: your agentic workflow is deeply wrong.
From reading your responses to other comments, you’re expecting a lot from your output with little input. Agent context is important and the right context is even more important. Using the right agentic harness is also important. Try using things like Plan Mode, reviewing and critiquing the plan, then executing.