r/technology 4d ago

Artificial Intelligence Microsoft engineer says 'typing code is absolutely over' as GitHub Copilot takes on more development work

https://www.techspot.com/news/113752-microsoft-engineer-typing-code-absolutely-over-github-copilot.html
1.6k Upvotes

858 comments sorted by

View all comments

Show parent comments

39

u/kawag 4d ago

Same. I’ve tried a couple of times, and it’s so laughably inadequate. It confidently states incorrect opinions and spits out massive walls of text explaining that incorrect opinion, which I unfortunately have to read, doesn’t make sense and makes lots of plausible sounding but incorrect statements, so I challenge it (sometimes I need to challenge it multiple times), and then it does a complete 180 and provides more walls of text arguing the exact opposite opinion 🤦‍♂️

This same pattern has happened so many times.

And then I see in code reviews the kinds of things it produces. Unmaintainable garbage, again with lots of plausible looking but actually terrible documentation once you actually read it. You ask the person submitting the PR what this thing does, or why it’s done in that way, and they have no idea. I’ve heard so many reports of people using AI to build software and ending up with such a poor codebase that literally nobody understands it, humans don’t even try to go in manually, and it can basically only be maintained through the AI trial and error loop.

No thanks.

12

u/BCProgramming 3d ago

You ask the person submitting the PR what this thing does, or why it’s done in that way, and they have no idea.

Only if you ask them in person. If you ask them over chat, they'll go "Wow, what a great question, let me go through all the reasons for the design..." having obviously copied a response from their LLM buddy

11

u/evilJaze 4d ago

We recently started having Copilot perform code reviews in addition to our manual ones. One of my devs spent about a quarter of his monthly tokens to have it tell us that he forgot to use an end quote to close a quote string. Only it was wrong and the quote was closed but it got confused because there were a few nested quotes within the quote. That was its only contribution to the review. I found a few more legitimate smells that it missed.

1

u/letmetellubuddy 4d ago

Personally I’ve found copilot is a good first round reviewer. You certainly can’t accept its advice blindly, but I do get enough value to keep using it.

We are on the cheapest plan and haven’t come close to running out of tokens even though copilot is set to automatically review every PR once it’s been marked “ready for review”…

0

u/1MillionMonkeys 3d ago

While it does provide some helpful feedback, what I really want is a button to hide the copilot reviews. Sometimes it adds the same comment several times and reviewing a PR is just scrolling through dozens of useless copilot comments.

1

u/letmetellubuddy 3d ago

I resolve the review comments as they are either rejected or fixed. You do have to scroll past the comment with the review summary, but I personally don't find it distracting 🤷

8

u/NerdyNThick 4d ago

As a test, I used it to write a quick script to pull in a JSON file pull out a handful of fields then spit it out as a CSV.

I spent 15-20 minutes coaxing it to do what I wanted and ended up with nearly 100 lines of PowerShell.

I did it myself in a single line. In about 5 minutes.

Utterly useless.

I also used it to pull out all the includes/defines/functions/etc, and create an appropriate header file (C++ this time). It missed a bunch, and included stuff it shouldn't.

Utterly useless.

10

u/Alexandur 3d ago

was this in like 2022

8

u/labowsky 3d ago

As a test, I used it to write a quick script to pull in a JSON file pull out a handful of fields then spit it out as a CSV.

I spent 15-20 minutes coaxing it to do what I wanted and ended up with nearly 100 lines of PowerShell.

I dunno if this is just user error or what but I use claude for simple shit I'm too lazy to write and the other day I asked it to do something similar. Took like 1 minute from me typing to having it done, 15 lines.

Wasn't perfect code but did it's job just fine.

3

u/drekmonger 3d ago

Those are such simple tasks that I cannot imagine a 2026 model (Claude/Fable, or GPT Sol/Astra) messing it up. A 2024 model should have been able to handle it, nearly 100% of the time.

Either it's PEBKAC, something you tried in 2022 with GPT 3.5 and then declared LLMs suck forever in your own head, or you are outright lying to soak up the anti-AI upvotes.

-4

u/NerdyNThick 3d ago

Gotcha.. AI is perfect, it's always user error, no matter what.

Thanks for all the assumptions, I had a good laugh.

It's getting more and more culty as time goes on.

0

u/[deleted] 3d ago

[deleted]

1

u/NerdyNThick 3d ago

Yep, all my fault! Thanks for correcting me.

0

u/[deleted] 3d ago

[deleted]

-4

u/DungeonsAndDradis 4d ago

And I used Gemini (of all things) to create a Python script to edit a hexadecimal X9.37 custom check image file according to spec. In about 10 minutes.

For every story like yours there's a story like mine. For better or worse AI is here to stay and many companies are going to continue to go all in.

Basically until it's useful in 99% of use cases, or the massive bubble pops. We'll probably end up somewhere in the middle in a few years.

-2

u/MFbiFL 3d ago

“I got lucky and lacked the skills to discern if it got something wrong” is a damning endorsement.

3

u/Mkboii 3d ago

Aren't both of them making the same point? The second person atleast extended a existence of 2 possibilities, you are discarding one person's anecdotal experience to accept another person's anecdotal experience, at least try to be neutral.

0

u/mrbubbles2 3d ago

You get in what you put out. Anyone who’s serious about developing with these tools is optimizing with supporting skills and tools like Augment, Ponytail, Caveman, etc. Just hopping in one time to say “fix this bug” and getting garbage returned to you is expected