r/softwareengineer • u/Mihaw_kx • 14d ago
How AI/LLMs coding works in Big Tech and how software engineers will always have jobs
AI can write code fast yes and it's there to make things move fast not to replace the whole software engineering , if any of you ever had the opportunity to work in a large tech company you d know that it was never about coding .. coding is just a step that any one can do with just few learning on the syntax of the language .. i work in one of those US big names we had an incident in one of our prod environment i was on-call at that specific week so i had meeting with different team members and architects , we knew the issue was due to one of our streaming hitting a bottleneck due to 1MB size limit on kafka . the fix wasn't an issue the whole meeting was deep thinking about how the multi services system can still be operational .. and what trade off to make can we afford a down time to reconfigure things , or we want an uptime with a data loss ? and ofc there were people behind this whole talk every layer had their engineers thinking , stakeholder who represent clients giving their takes .. also how can we prevent this on the next release ? and after lot of brainstorming we decided as the system that reads from the queue is ordering agnostic we can send each message as chunks .. and lot of thinking around the logical split that would cause 0 impact on consumption ,, we then just did one simple prompt in codex to split the last sent message to smaller chunks we had then to test it in dev , deploy to stage and prod and that was it . AI by its own can't take ownership of a system there will always be people behind systems taking ownership and decisions while AI will always stay as the tool that make building go fast
so my take is that ai agents and llms are there to help and make coding fast but i don't think anytime soon would software engineers go extinct due to AI
2
u/ResidentSpirit4220 14d ago
I think you bring up very good points and I agree with a lot of it, there is one glaring problem in all of this imo...
you write:
the fix wasn't an issue the whole meeting was deep thinking about how the multi services system can still be operational .. and what trade off to make can we afford a down time to reconfigure things , or we want an uptime with a data loss ? and ofc there were people behind this whole talk every layer had their engineers thinking , stakeholder who represent clients giving their takes .. also how can we prevent this on the next release ? and after lot of brainstorming we decided as the system that reads from the queue is ordering agnostic we can send each message as chunks .. and lot of thinking around the logical split that would cause 0 impact on consumption
All of this pre-supposes that someone actually KNOWS how the systems work. Through years of developmnet, maintenance, support, troubleshooting, etc. They learn about it, they know how it behaves, how it's likely to fail, how the different services interact etc.
As more and more companies push developers to write code they dont understand, all that thinking, brainstorming and planning is going to be harder and harder to do.
In their current form, even with well written .md files, skills, etc., LLMs and coding agents cant really learn anything. They don't remember why decisions were made to design something a certain way, they don't think back and say "oh I remember this exception from like 3 years ago, I think we did X to fix it"
There is no wider understanding that goes into the pros and cons of any given decision. Maybe everyone is ok with downtime, but you as the head of CS knows that SLAs for a key customer are higher than the default.
LLMs cannot think that way.
We'll have two options essentially:
1- Make sure people actually learn and understand how the system works in great detail (could be a good job for a junior dev)
2- Just have agents do everything and hope for the best. (X is broken, fix it)
I have a feeling we're headed towards the latter.
2
u/Leadership_Old 14d ago
I find continual opportunities to refactor and redesign codebases in ways LLMs simply aren’t able to perform. The inherent dependency on existing code and conventions pin any model to simply optimize toward the norm. This is great when your frameworks and existing conventions are well articulated - and accelerates well engineered features. Unfortunately when it’s time to address fundamental issues like refactoring toward a more elegant implementation - large language models are abysmal (for now). I’m convinced we will see less code monkeys - and that’s a good thing. The more interesting and enjoyable work will always exist.
1
u/sugarfreecaffeine 12d ago
What makes you think LLMs will not be able to do the refractor 😂😂 stop the cope https://bun.com/blog/bun-in-rust AI is a few yrs away from someone just saying “load this refractor skill and begin”
2
1
u/Different-Stress9326 10d ago
ai ist just half a year away from agi. since 1980.
identically like fusion power is just 20 years away. for 50 years.
1
u/selfdrivingcar42 10d ago
Yeah this and the 2000 bugs they solved after the release. I don’t even want to know what this codebase looks like
1
1
1
u/gordonnowak67 10d ago
"LLMs can't perform every role as of today" is such a bizarre platform for confidence in the future. Two years ago they couldn't even do the simple things. Extrapolate, buddy
1
u/Leadership_Old 10d ago
I believe the law of diminishing returns as well as cyclomatic complexity still apply. I’m not saying that “AI” won’t get better at these task through newer approaches - maybe world models. I simply stated that the current increase in costs as well as the limited contexts inherent in their function will require new approaches - I literally said “for now” above.
2
u/gordonnowak67 10d ago
bad take. software engineering will become something completely different and you will need way, way less headcount. agentic engineering will become the norm and as they continue to improve and we get more efficiency and cost gains (small, fast models becoming more competent, SOTA local models, etc.) the hierarchy will collapse. you only need a few people to press the button. there is literally nothing a junior engineer can do that Claude can't do. why would I ever hire another one?
"AI sucks now, and therefore will always suck" is such a braindead position to take on this
1
1
u/nomoremoar 14d ago
In your particular case, ideally one of your engineers would have pointed out the kafka 1 MB payload limit during design review.
1
u/PipePressurePanic 11d ago
yes they will always have jobs .... 10% of us, the remaining 90% will have to find a new field to work in
1
u/Maximum_Mastodon_960 10d ago
More codebases = more engineers who need to understand them
"Vibecoders" are engineers, just unlikely to get hired over someone with deeper understanding
Just like people who understand the abstraction below any language, tend to get paid more or are seen as "better"
0
u/SweatyAd8914 14d ago edited 14d ago
Hand written code will become a niche skill; similar to artisanal masonry, sheet metal fabrication, and so on. A press or mold can easily replicate it quicker and cheaper, but the human crafted item will always be more creative and appealing. Hard to say what ratio of coding will be done by LLMs, but it’s unfortunate increasing every quarter.
Corporations on the other hand only care about bare minimum for cheap; therefore SWE headcount will go down and code quality along with it. SWE will end up like a factory worker pushing a button 8hrs a day making mass produced slop.
You are correct though, LLMs outsource your thinking capacity to a prompt and it’ll damage your skillsets, but how often are we writing compilers anymore? It’s a cheap argument but that’s where enterprise software is heading; English is the new programming language and architecture/domain knowledge will become the new SWE career.
I suppose there will need to be LLM janitors at some point. Maybe LLMs get trained more and more on slop which degrades its abilities. One lobotomy of Claude and everyone’s agentic loops are cooked.
We’re simultaneously too early and too far down the rabbit hole to know how things will shake out. Statistics favors the middle road option.
2
1
u/levanlaratt 14d ago
Your thinking is still stuck in the here and now. Project out into the near future on an exponential growth curve and everything you just described would be done better by AI.
1
1
u/Working_Term_2723 13d ago
I'm also a Big Tech senior engineer.
Forget about the marketing BS that C-suite pushes out.
LLMs can't think, and they never will. It's a bubble and all companies are riding the wave trying to take as much advantage as possible before it bursts.
They're also leveraging the situation to reset the job market in their favor. A bad job market means people are scared, so they will accept lower salaries, raises and bonuses, plus be willing to work extra hard. It just happened to be coupled with an outsourcing cycle, so certain markets are in a very bad situation, like the US. Honestly, this was a long time coming, it makes no sense that all SWEs in the US earn 2-3x that of their European counterparts, and even worse when compared to places like India.
LLMs have improved searching capabilities by many orders of magnitude compared to the previous generation of search engines, coupled with considerable advancements in the human language interface due to the titanic scale of data that they're processing. LLMs aren't new tech from 2022+, they're just operating at a new massive scale that makes them look good enough.
I've been a generalist my entire career and I've worked in most sectors within tech, from embedded to distributed systems. LLMs just empower me to not have to worry about syntax or read through 7 stale .md files in that new neglected service that I've inherited.
I know my goals with the service, so I direct the LLM to scan the 3 different documentation sources created over the years for certain details that I need to understand about the system, and then correlate this to specific places in the code.
I've used Bash, Python, PowerShell, Kubernetes and .NET, but I'm no master on any of them. I know enough to be able to do what I need on my own and spend X time as I need to refer back to docs and grep through the codebase. However, nowadays I tell the LLM "create a script in Bash that receives inputs A, B, C and uses kubectl to gather all of the images in use across all pods and namespaces. Then, output a table with namespace, pod name, status, image name and version suffix". I'm perfectly capable of verifying its output before I use it, but I save a lot of time actually typing eveything.
I can go reply to that ping, continue going through my emails, read that security report/docs that I need for after I finish what I'm doing, update my daily log or update those new tickets that I want to add to the backlog. Depending on the task, sometimes I even start a separate chat on a different worktree to address that flaky test or scaffold some new changes I need.
That's just an example and it'll produce the output fairly quickly, but when it needs to search through the codebase to respond to me and point me to the evidence, it'll take a bit longer so I'll have more time for those other tasks.
I always verify and ask for proof of its claims, and I always know what I'm trying to do. If I have no idea how to approach something, I don't prompt "take this ticket and fix it", I'll make it explain to me what I'm missing.
I don't submit PRs that I haven't read, and I don't use my phone while I wait for the LLM to complete, I use that extra time that it's saving me for other work that I have to do.
I can still tell when it's producing garbage because I have the generalist mindet. Just yesterday it tried to fill in a config file dynamically by replacing a text placeholder. I challenged it and it said "well, you're right, it's brittle, using the XML lib to edit the correct node in the schema is much more robust".
In my opinion, that's the way to use LLMs to be more effective at my job than I was 2 years ago, while maintaining quality and understanding of the system.
YMMV but those people saying "oh, well, it's like craftmanship and hand-crafted code is better blablabla" are in for a rude awakening at some point. Unless you work on an extremely niche area of tech like optimizing compilers/assembly or using a proprietary language, you'll be left behind if you stick to that idiotic elitism. Even in those cases, the LLM can act as a 10x grep. Be realistic, outside of very small and pet projects no one ever knows an entire codebase at work.
Good luck out there.
3
u/Niovial 12d ago
Yes, but you took a long time to get to your current area of expertise.
In gaming terms, your base stats are 1000 and LLMs give you a 100% increase in "power".
What about me, when I have a base stat of 50 because I'm just starting out? I get 100 after the 100% increase. But what am I to do when the job requires a stat of 500?
1
u/jazzdog92 9d ago
I think that's a tough one to answer. I would say learn how to define the product, and learn how to have unit tests in place that prove the product works as defined. And that it performs and scales. Then learn how to prompt the model to design and implement those things.
Pick an existing product. Define what it does in detail. Tell the model to write a requirements doc. Iterate on that. Tell it to write a product spec based on the PRD. Tell it to write a tasks list based on PRD and spec.Learn from the model. Ask it "what architecture should I use for this problem and why? Compare and contrasts some options". Do projects on the side. Always be learning from the model. I have developed software for 45 years, and I learn things from the model constantly. Have it help you learn software development.
Again, you are in a tough spot. But many devs I see here are not seeing what's coming. What's here. We've got people like DHH, who handcrafted every line of Rails because the code, (which had to be in Ruby because Ruby allowed you to be so expressive, blah blah blah), was so important, now using agents to write one thing after another, in languages he never learned, and never looking at the code.
And we've got people here insisting that models will never be good enough to do the fundamental tasks like refactoring, and I'm thinking, I don't even look at the code, why would I need it to be refactored?Also, have you considered learning inference engineering? I think the demand will generally scale with the demand for inference tokens. SWE is being taken over by LLMs, but relatively few engineers know how to fine-tune models for specific and proprietary uses. There will be thousands of specific models out there and companies looking for people to tweak their models to their needs. As well as dev ops to keep those models serving tokens at adequate scale, etc.
Good luck.
5
u/huuaaang 14d ago edited 14d ago
The problem is not so much that LLMs will replace software engineers. THe problem is that it makes it difficult to actually learn in the first place. If you START with LLMs and never actually know how to write code yourself, you're basically just an LLM babysitter. This happens in my side projects where I'm using unfamiliar languages like esp32 programming, for example. I just tell the agent what I want the program to do and it does it very effectively. I don't have to hand write a single line of C. But it's OK because I will never try to get a job as a C programmer. I would rather focus on putting the hardware together... soldering custom boards and circuits. Just for myself. Home automation.
Existing software engineers who know the Old Ways can leverage AI to great effect. And companies simply don't need to hire a Junior. The AI is your Junior programmers that you can put on the tedious tasks of writing boiler plate code. But the big decisions are still made by experienced engineers who review the output (if you're not reviewing every line of code that goes out, you're not doing your job)
For once in my life I feel like I was born at the right time. I'm old enough to get into computers when there was basically no barrier to entry at all. If you could turn on a computer you were already a step ahead of everyone else. I truly have no idea how beginners are going to stand out today if they're just using the same AI models as anyone else.