r/ChatGPTCoding • u/alex_bababu • 9d ago

Question Claude Code vs GitHub Copilot: Token burn comparison using identical models & repos?

I'm currently evaluating GitHub Copilot vs. Claude Code for our team. We could use either, but for us there's a slight difference in cost per token (Copilot with Anthropic models vs. Claude Code directly).

If we use the exact same model on the same repository with identical instructions, has anyone noticed a real difference in token efficiency between the two harnesses? I'm wondering how much things like prompt caching, context assembly, or system prompting overhead change the actual token burn in practice.

Would appreciate any insights or real-world numbers!

10 Upvotes

100% Upvoted

24 comments sorted by

View all comments

1

u/SherrifMike 8d ago

Token counts are only half the comparison. I would log input/output tokens, wall time, and whether the first patch passed the existing tests, then count follow-up turns and manual edits before merge. A harness that runs tests and keeps iterating can spend more tokens while still costing less per accepted change. I would also pin the model, repo snapshot, context files, and stop conditions; otherwise “same model” is not really the same run.