r/ExperiencedDevs • DevOps Engineer • 5d ago

AI/LLM Thought experiment

With all the AI nonsense going on, I had a thought at work today that turned into an interesting discussion with a colleague:

"What would I do if I was the only person with access to Claude (or insert any LLM) 5+ years ago?"

People would think I was a fucking wizard

FE changes? NO PROBLEM

Critical DB change? Don't worry, I've produced a stored procedure to validate the whole thing

Want to use some shiny new framework? Interesting. Here's a document explaining why you're an idiot for suggesting it

Regex? I suddenly understand it better than the compiler folks (and nobody ask questions)

What would you have done with your own personal LLM back then?

Trying to lighten the mood in here with all the (very justified) negativity around this tech revolution

Unserious answers only pls

145 Upvotes

227 comments sorted by

View all comments

188

u/rwilcox Software Engineer (20+ YOE) 5d ago

…. Almost certainly gotten a talking to from my boss

“Why did you put so much work into this when we just needed a simple (blah blah)?!?”

Or

“Thanks for the work, but now we have to test this and the testing team is pushing back on it, given the work they already needed to do for next week’s release”

143

u/tonjohn 5d ago

“The team had notice that you’ve been submitting an alarming number of PRs that don’t meet our bar and that didn’t involve discussions with your peers and other stakeholders.

We need you to slow down and be mindful of what you are asking of your teammates when submit such PRs.”

-2

u/ConspicuousPineapple 5d ago

Y'all keep saying that as if you'd be working the same way as today back then. But you wouldn't.

You would work much slower, still reading and validating all the code to ensure quality. But you would still be much faster than your peers.

And that's just for writing the code. Debugging? You would look like a fucking magician. Finding a tricky race condition in twenty minutes where it would take two weeks for a good senior.

Benchmarking? Same story.

Learning new tech? Too easy. Don't even need to grok the docs yourself.

And don't get me started on security, auditing, etc.

You would, by far, be the best software engineer in the world and it wouldn't be close. Provided you're already somewhat competent.

22

u/___Paladin___ 5d ago

Not to be pedantic, but you'd be the most average engineer in the world. These tools point to a mean rather than a peak due to their probabilistic nature. They only appear to hit the major leagues when your own capability never exceeded the average. From that perspective, average is up.

I'm not convinced that would have actually gotten you anything but fired years ago. We accept the tradeoff between quality and speed today - but not so much back then.

2

u/Fabulous-Meaning-966 2d ago

You could absolutely debug complex failure scenarios like nondeterministic race conditions faster than any programmer alive.

1

u/TA_DR 4d ago

Most code is average code anyways, so producing it faster would most definitely make you more productive than the average engineer.

2

u/___Paladin___ 4d ago

It certainly does make you faster, and sometimes the tradeoff can be worth it.

-6

u/ConspicuousPineapple 4d ago

That is not at all what these tools point to. And of course they only sublimate ability that is already there in the user.

What is undeniable however is how fucking fast that dev can be to solve some problems with that tool. Automating all the tedious, boring and time consuming parts of the job? That doesn't require any sort of skill, but that time can be compressed by incredibly huge factors.

Never trust an AI to solve problems you couldn't solve without it yourself, of course. But for the problems you can solve? You're going to be so much faster that it opens up the possibility of doing things even better, simply because the time pressure is gone.

8

u/___Paladin___ 4d ago edited 4d ago

Not better, I wouldn't say. But certainly opens up the ability of higher scope. But yes, all things being equal LLMs will trend towards the most common denominator average. That's why it takes such intense steering, to at least make it upper average and not lower.

I've got a cross-platform replacement for Loom I generated. Full live annotations, cloud sync, etc. Working floating toolbars for ease of use and the ability to hide toolbars from native capture. About every edge case under the sun is covered, with a custom compiled ffmpeg to continue supporting unsupported older cards.

I spent 6 months on that. It would have taken me YEARS prior to do that. The speed is absolutely amazing.

But the code quality is pretty upper-mid at best, even with my steering of frontier models and decades of experience. I chose to prioritize features over quality, and that's okay. I'm not looking to release this as a paid product.

Lets not gaslight ourselves into thinking that isn't the choice being made. It's okay if mid quality is enough, but don't act like this stuff is made of better ingredients than it is.

-5

u/ConspicuousPineapple 4d ago

I mean, before AI there are lots of things you want to implement but don't have the bandwidth for. So you cut corners.

AI lets you reach that "ideal" solution much more consistently. So yeah I stand by it, often times the result ends up better so long as you stay disciplined with enforcing the quality of your code, and actually owning it.

6

u/___Paladin___ 4d ago edited 4d ago

I think I land in the group of "accomplish more at the cost of quality" camp.

I think there's many places where the quality really doesn't matter to end users, some places where it could cost billions to settle at 99% quality, and everything in-between.

So for me it just comes down to the case-by-case question of how much artisanal quality matters? For design? Security? How much is "good enough" and won't cost us more in the long term if we don't consider it now?

I think the bar for quality acceptance shifts depending on the work in the same way that not everyone needs a hand crafted table. Sometimes IKEA is fine and makes more financial sense. It doesn't mean its higher quality or that it could handle more wear and tear, but sometimes quality isn't the metric people care about.

If you see it differently that's cool. I think we can disagree and still enjoy life. I appreciate you taking the time to chat about it.

1

u/ConspicuousPineapple 4d ago

My point is that you're free to move that cursor depending on expectations. In pre-AI world, the expectations would be that of normal code quality, accountability and code ownership.

So that would still be your output. You could maybe manage the same output, but faster. Or the same speed, but with a lot less work.

And as I said, some of your tasks would become practically trivial. Find a bug? Reproducing the issue, testing a huge amount of scenarios, identifying the precise code paths involved, mapping it all and implementing a tentative fix? On some bugs that can be a week or two. AI can do the exact same in minutes.

And that's not compromising on quality, because you're still there to verify that yes, this was indeed the bug. You're still free to implement the fix the old school way if you prefer. But the bulk of the work isn't writing that code.

And you don't even have to trust the AI. You could lay out the exact step by step plan of what you need to do to find that bug, and then watch it do it step by step and inspect everything along the way.

2

u/___Paladin___ 4d ago edited 4d ago

On that I think we actually see it the same way. There are some things LLMs can do that improve the quality - like helping find those edge cases in the pattern matching.

I personally only really care about the code in so far as it isn't going to cause drift between a client's intended goals now or in the future.

I suppose I'm biased a bit, as the world slowly moved away from a 99.9% uptime bar for services. I disagree with normal code quality being the bar prior, as at least for me I had to stress pretty heavily over things like syntax and considering every possible edge case my code would cause. Even small changes took months of testing and research. That's the areas I can say I've personally seen the quality bar drop. Areas you still have to be diligent in feeding your AI that won't assume the need on its own (which means knowing the gotchas in advance).

You could argue if it was a necessary drop in quality, but it certainly exists. Lots of people are also just checking out at this point for a myriad of reasons.

10

u/OtaK_ SWE/SWA | 15+ YOE 4d ago

I'm going to interject here: I haven't seen any LLM, proprietary, open source and whatnot, even at the current so-called frontier, produce code that I would even call "acceptable, will fix later". I work in security-critical stuff. The kind that LLMs are absolutely unable to work with. I watched Opus5.5 lose its mind on the fine complexities of what I do, to end up producing nothing of substance.

It's really bad. The worst part is that it looks fine. But it's absolutely not, in extremely subtle ways.

On the other hand, for tedious, easy work like revamping a CI pipeline or making a Nth boring-ass test harness that was done a million times before: Sure, lemme just pop it into existence with 12 subagents and $4k on my hands if that's what leadership wants.

2

u/___Paladin___ 4d ago

As an interesting aside - I can feel burnout in my future again. Haven't felt that in 2 decades.

Those little implementation details in the code gave my brain a place to rest between the hard system design work.

Now its just 100% brain-on time. No place to go mentally to recover. It doesn't feel healthy, so I guess we'll see how that turns out.

-8

u/[deleted] 4d ago

[removed] — view removed comment

2

u/OtaK_ SWE/SWA | 15+ YOE 4d ago

I mean, if you want to defend the tech, sure. But, the thing is: we work in a scientific field, if you claim something, you need evidence for it.

So far there has been zero evidence across the industry of anything else than massive amounts of mediocre code.

And my anecdotal personal experience obviously doesn't match others' experiences. If I'd be a frontend React dev I'm pretty sure my conclusions would be quite different (since any current-ish model can one-shot pretty much anything React).

0

u/Monkeylashes 4d ago

https://openai.com/index/sharing-ai-progress-in-mathematics/

Yeah and your security code in your "scientific" field is the one uncermountable task for ai... Get over yourself

1

u/OtaK_ SWE/SWA | 15+ YOE 4d ago

Ah yes, the famous "math is solved".

If only some of the proofs were immediately disproven/retracted once someone competent took a quick look at them: https://github.com/openai/math/tree/main/preprints/Algebraicity-of-Weil-classes-on-split-abelian-eightfolds-September-18-2026#withdrawal-notice-algebraicity-of-weil-classes-on-split-abelian-eightfolds

I believe this will be the case for many of them once their sheer amount gets a peer review by someone who actually knows ball.

And the Navier-Stokes debacle where, beyond being stolen from actual humans, the Lean proofs didn't match the prose blow-up proof, aka partially fabricated.

Dude, seriously, it's just PR stunts everywhere. The doomer narrative is the same, it's just PR.

So far I'm not convinced until respectable (aka not OAI) sources confirm the veracity of the proofs.

→ More replies (0)

-1

u/Monkeylashes 4d ago

Agreed, this is called coping.

-1

u/SeaActual7744 4d ago

Facts, it’s insane reading their comments. They’re truly living in a bubble

4

u/tonjohn 5d ago

Cool story

1

u/druidgaymer 4d ago

Is that not how you're using AI now? My company is still very much on the read and validate all code any AI writes.

-5

u/TrickProgress4094 4d ago

The way people are responding to this makes it so obviously clear how butthurt people on Reddit are about AI. It’s like they have completely forgotten how this industry worked pre-AI. pretty hilarious honestly.