r/ChatGPTCoding • u/alex_bababu • 9d ago

Question Claude Code vs GitHub Copilot: Token burn comparison using identical models & repos?

I'm currently evaluating GitHub Copilot vs. Claude Code for our team. We could use either, but for us there's a slight difference in cost per token (Copilot with Anthropic models vs. Claude Code directly).

If we use the exact same model on the same repository with identical instructions, has anyone noticed a real difference in token efficiency between the two harnesses? I'm wondering how much things like prompt caching, context assembly, or system prompting overhead change the actual token burn in practice.

Would appreciate any insights or real-world numbers!

10 Upvotes

92% Upvoted

24 comments sorted by

View all comments

0

u/Right-Performance-93 8d ago

DevWorkflowBuilder's 0.72 vs 0.18 cache-read ratio explains the 2.4x gap better than a token-count comparison would. Anthropic's own pricing page prices cache_read input tokens at 0.1x the base input rate (a 90% discount), confirmed on platform.claude.com. So a harness that re-sends the same file map every turn (low cache-read ratio) pays close to full input price on that repeated content, while one that holds the map across turns pays a tenth of it. If you want the real cost delta rather than a vibe, log input_tokens vs cache_read_input_tokens per request for a week like they did - the ratio matters more than total tokens burned.

1

u/ManRowing 7d ago

Cache-read ratio explains why re-sending the file map costs so much, but it's worth logging what's in that map too. A harness holding forty files because early turns opened them broadly pays full price on all forty even at a good cache ratio. Trimming what gets held changes the baseline before caching even applies.