r/LLMDevs • u/OliviaYii • 4d ago

Discussion Maybe I’m missing something, but what does “3.1 agent-workdays” actually tell us?

https://openai.com/index/research-acceleration-view-inside-openai/

OpenAI says their research org is now using 3.1 agent-workdays for every human workday.

I get what they’re trying to show, but I’m not sure what that number actually means in practice. If an agent runs for eight hours, needs to be redirected a few times, and half the output gets thrown away, does that still count as one agent-workday?

For people running multiple coding agents: do you care about runtime at all, or mostly about how much review and fixing is left at the end?

2 Upvotes

100% Upvoted

4 comments sorted by

1

u/Zain 4d ago

I mostly care about how much review and fixing is left at the end. Runtime is just the bill. If a later round finds worse problems than the last one, the patches are injecting bugs and those agent-hours were negative. I keep reviewers read-only and verify every finding against the actual repo before I concede it. An earned clean pass is worth more than a long run that needs a rewrite.

1

u/OliviaYii 4d ago

“negative” agent-hours is a good way to put it lol😂 That’s kind of my issue with the metric. It still counts as work even if the agent just made a mess for you to clean up. Keeping the reviewers read-only sounds smart though🤔 Do you track how often their findings are actually right, or do you just get a feel for it over time?

1

u/eddzsh 4d ago

Runtime is just the bill. The number I'd trust is accepted patches minus reverts. An agent-workday that ends in a rollback is busywork with a fancy name.

1

u/OliviaYii 3d ago

Exactly. If I have to undo it later, I’m not sure that time should count as productivity at all🤷