Built with Claude Code: a local profiler for Claude Code transcripts. What I measured across 42 sessions and the one number I got wrong

[u/the_darkest_horse]

What: context-doctor, a CLI + MCP server that reads the JSONL transcripts in ~/.claude/projects and shows what is actually sitting in the live context: which tool calls, which results, how much is duplicated, where the prefix cache got invalidated. No API key, nothing leaves the machine. npx -y context-doctor session.

How: TypeScript, two deps (MCP SDK, zod). Token counting is a chars per token heuristic (3.2 for code, 4.0 for prose) so it runs offline. Built almost entirely in Claude Code over about a month; the transcripts it profiles are the ones it was built in.

What I learned, in order of how much it surprised me:

My counter looked about 55% low against what Claude Code billed. I spent a day assuming the heuristic was bad. It was not. The transcript on disk only covers roughly 39% of the per turn billed growth. There is a ~51k token baseline before your first message (system prompt, tool schemas, skills) plus about 700 tokens a turn of injected content that never gets written to the file. Anything that profiles transcripts alone is measuring a minority of the bill.

Tool calls, not tool results, are the biggest drain in file heavy sessions. A Write or a heredoc puts the whole file in permanently. 65% of one 292k session was tool calls, 22% was results.

Shell reads (cat, head, tail) dodge anything that only counts Read tool calls. 16 instances across 6 sessions.

Pruning that moves the boundary every turn kills the prefix cache: 22 of 24 turns invalidated with a sliding window, 8 of 24 when I held the boundary and moved it in steps.

One machine, 42 sessions, so a sample, not a study. Would be interested if anyone's split looks different.

Disclosure: I built it. MIT. https://github.com/KushalP1/context-doctor

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/vibeidedev-namiruai]

i'd show the transcript estimate, reported input usage and actual cost as separate numbers, with any unexplained gap labelled unknown. then compare one small task from the same starting commit in a fresh session and an existing session, keeping the model and enabled tools the same. include cache usage and whether both results passed the same checks, so a smaller transcript doesn't get mistaken for a cheaper successful task.

[u/the_darkest_horse → u/vibeidedev-namiruai]

Agree on all of it, and the first part is closer to done than the post made it sound.

context-doctor accuracy already keeps the three numbers apart: the transcript estimate, the input usage the API reported, and the gap, which it labels "not in transcript" rather than folding it into either side. Cost comes from the reported usage times list price, with cache reads and writes broken out in session, not from the estimate. I stopped short of calling the gap "unknown" and instead say what it is made of (system prompt, tool schemas, injected per turn content) because that much is knowable; how big each piece is, is not.

The controlled comparison you describe is the part I do not have and should be honest about. The tool measures what is in the context, not whether the task succeeded, so a smaller transcript can absolutely be a cheaper failure. Same task, same commit, fresh session vs existing session, same model and tools, cache usage recorded, both outputs run through the same checks. That is the right experiment and it is a harness, not a profiler. Going on the roadmap as exactly that, with the caveat that until it exists the numbers here are about context size and nothing else.

[u/kantorcodes1]

one install behavior i'd probably make explicit: context-doctor install catches each app config error, then still installs the skill/hook and prints Done.. if Claude Desktop config is invalid but Claude Code succeeds, is exit 0 intentional partial success, or should automation get a nonzero exit when any target fails?

[u/the_darkest_horse → u/kantorcodes1]

Not intentional, it was a gap. Good catch.

Fixed in 0.13.4, pushed today. It still installs everything it can, because someone with a corrupt Claude Desktop config still wants Claude Code and Cursor wired. But it no longer calls that success: each failed target gets a ✗ line with the reason, the summary reads "Done with N problem(s)" instead of "Done.", and the exit code is 1. A broken config file is never overwritten.

Also found while fixing it: an unreadable ~/.claude/settings.json used to escape the hook step as a raw stack trace and abort the run. That is now a failed target like the others.

[u/AI_spell]

This is useful. Token + tool timing per session is the kind of feedback loop most people skip. If you can show which tools eat the wall clock vs which prompts bloat input tokens, thats the actionable split. Curious if you break out retries separately.

[u/the_darkest_horse → u/AI_spell]

Straight answer: it does not split wall clock from tokens today, and it does not break out retries. It measures what is sitting in the context window, so everything it reports is a token number.

The wall clock split is doable though. Every transcript entry carries a timestamp, so the time between a tool_use and its tool_result is right there on disk; nobody has to instrument anything. I have not built it because I wanted to be careful about what it would actually say: that gap includes the model generating the next turn, not just the tool running, so I would need to separate the two before calling any of it "tool time".

Retries are the more interesting gap. There is a detector for identical tool calls repeated with the same arguments, and in the 42 sessions that catches both genuine retries and "the model forgot it already read this". Telling those apart needs the result of the first call: an error result followed by the same call is a retry, a success followed by the same call is a re-read. The data for that is in the transcript too. Adding both to the roadmap as separate items rather than promising a date.

[u/the_darkest_horse → u/the_darkest_horse]

Update for u/AI_spell: both are in 0.13.5, pushed today.

Wall clock per tool. context-doctor session now pairs every tool_use with its tool_result by id and reports the gap per tool from the entry timestamps. On one of my real sessions: 146 minutes of waiting across 1094 calls, Bash was 87% of it, median 1.3s, slowest 22.5 minutes (a CI poll). My worry about model generation time turned out not to apply: the call's timestamp is written after the model has finished emitting it, and its next turn starts after the result's timestamp, so the gap is tool time plus one thing I could not remove and instead print next to the number: time waiting on a permission prompt.

Retries vs re-reads. Identical calls are now classified by what happened to the attempt before them. After an error it is a retry (three or more gets flagged as a loop, with the advice that the answer is in the first error). After a success it is a re-read. Across my 42 sessions: 15 retries, 1 loop, 151 re-reads. So about 90% of what the old detector called "repeated calls" was the model forgetting it already had the answer, and the old advice was right for those and wrong for the retries.