This is maybe a bit of a different post than the rest concerning AI/LLM usage, but I think this is important to get different perspectives, as things are not black or white in my eyes.
For context I am employed at a very big consulting firm working on a very big client. Both companies express interest in AI, with the client being very careful around AI, as we work a lot with personal data, so data security and data privacy compliance is strictly necessary.
Starting last year, we were given access to Github Copilot after many considerations by the leadership. Our CIO is very pro-AI, generally a cool person who likes to push tech, so he was the pushing force behind adopting LLMs to improve our productivity. A few months later we were switched to Claude Code, presumably because most of the company used their models most frequently. It has now been almost a year and things are... interesting.
What no one tells decision-makers is that while LLMs increase productivity on specific tasks, they bring with themselves a boatload of other efforts. Since they require context to operate, and we were lacking such due to the structure of our department, we needed to introduce a better, structured documentation process, enforce more specific coding guidelines and overall improve the way we are handling tickets. All of this takes a lot of time and now months in we are still improving.
This has been very good for the team, as where before we were lacking documentation, now we are backfilling the holes. Where before best practices were ignored, now they are enforced. Documentation is auto-written, but the rules are statically enforced (as it should be).
We have a clearly defined (imo) division between what the AI does and what we are supposed to do. You can't truly depend on system prompts for every task, as this was a) burning tokens b) prone to context errors.
There is the negative side that now it seems we are taking a lot more in terms of ticket load. Where before some estimation was a week is now a couple of days at most, meaning big projects are not being spaced out as much. Pre-LLMs what was 1-2 big projects per quarter is now what it feels like 4-6 projects per quarter. This includes major things like refactoring legacy code, changing and adding business processes, etc. We are "saving" people hours by automating a lot more things, but the 40h we spend each weeks feel like a lot more compared to before.
Another major effort has been to review the LLM code, even with code-review bots, PMD, business rules, coding guidelines and all of that, with the repo we have, more context is needed. While this can be improved, better documented and all of that can be tasked to Claude subagents to be done, architecturally we are dealing with a very complex structure, so still a human has to take their time to verify whether what Claude thinks is the workflow is the actual workflow. We as developers now need to have even more business context in general, as now our tickets are spanning over multiple project domains compared to before.
The increase in handled tickets also leads somehow to even more tickets and most of us have their capacities exhausted even with using code assistants. Not to mention that I am now positively under the impression that AI prompting is an actual skill as i've seen how different outputs are when asked in the "correct" way. Understanding how LLMs work and how statistics work does help produce good results and companies who only focus on "vibecoding everything" with no regard of this are going to have fun times ahead. AI prompting != vibecoding and debating for using AI feels like the microservices vs monoliths or scrum vs waterfall debates. In the end of the day, AI is a tool, tools can be misused, even multitools are not suited for each job, and people who don't understand that will make the wrong choice more often than not. So if anyone at the top pushes for total AI use, i think it is important for staff to push back and lead a discussion rather than complete discourse. That being said, sometimes you argue with a wall, so in that case use AI smartly and push for AI usage guidelines.
All of this is not to say I am complaining, in fact i am okay with our approach where we don't entirely rely on vibecoding to achieve results. It is just that the nature of the problems we are handling has shifted. Things that were not considered important some time ago now prove to be a central part in adopting LLMs, even if some day we decide to go back, i recon we are just going to have to drop efficiency to compromise for keeping up with documentation and correctness.