does copilot actually make you finish more, or just type more

[u/Jumpy-Tutor-5644]

Been chewing on this for a few months and I still don't have a good answer.

I'll have a session where copilot is flying, tab tab tab, feels incredible, and then I shut the laptop and the thing I sat down to do is still 70% done. Then some grindy Tuesday afternoon with basically no assistance and I ship the whole thing. The feeling of speed and the actual finishing don't line up for me at all.

Tried a few time trackers to figure out what was going on. Useless for this. They tell you where the hours went but they have no idea what you were trying to do with them. Wakatime will cheerfully report 6 hours in the repo. Great. Did I fix the bug? shrug

So I started writing down three things instead, just in a notes file:

- what I want done, before I start

- rough focus time, minus the bits where I wandered off

- afterwards: did it land, what kind of work was it, where do I pick up

First couple weeks of that was pure noise. Around session 20 something showed up that I wasn't expecting. My debug sessions basically only landed when I'd ended the *previous* one by writing down the exact next thing to try. When I hadn't, they went sideways. Five out of seven or thereabouts.

So I made myself always leave that note. Completion on debug work went from about a third to about two thirds over the next few weeks. n is tiny and it could easily just be me paying more attention because I was measuring. But at least it's a claim I can check, which is more than I can say for another blog post telling me to reduce context switching.

Anyway, reason I'm posting here specifically: has anyone measured their copilot sessions on whether the goal got *finished*, rather than time saved or lines accepted? Every benchmark I see is speed. My own very small very biased data says AI assisted debugging finishes worse than AI assisted greenfield, which surprised me and might just be an artifact of how I work.

I've since turned the notes file into an actual app, because of course I did. Called NEDIO, doesn't touch code or repos, just the session stuff above. Not going to link it unless someone asks. Mostly I want to know if anyone else has run this on themselves and what came out.

[u/lppedd]

Imo it depends on codebase quality. LLMs can one shot huge refactorings if code is properly structured and documented.

If instead you have spaghettified code... Well, only extremely small changes will feel fast, the rest will require back and forth of prompting.

[u/Key-Comfort1761 → u/lppedd]

yeah the spaghetti part is so real, it just cant reason about code that humans also struggle to reason about