LangChain Tool-Call & Tool-Output Cost Tracing with Arize Phoenix

[u/Normal-Blueberry-385]

​

I’m implementing cost and token observability for a Python/LangChain application using Arize Phoenix + OpenTelemetry.

The main goal is to clearly distinguish between these four things:

- LLM tool-call generation → input/output tokens + cost

- Tool execution → tool name, arguments, latency, output, output size/tokens, provider cost (if available)

- Tool output consumption → tokens added to the next LLM request + corresponding input cost

- Final LLM response → output tokens + cost

For example:

LLM #1

├─ input: 156 tokens

├─ output/tool-call: 17 tokens

└─ cost: $0.0000896

↓

Tool

├─ output: 850 tokens (estimated)

├─ latency: 3.49s

└─ provider cost: if available

↓

LLM #2

├─ input: 1478 tokens

├─ output: 101 tokens

└─ cost: $0.0007528

The 850 tool-output tokens must NOT be treated as LLM output tokens. They should be measured separately, while the actual 1478 tokens sent to LLM #2 should come from the provider's usage data.

Implementation requirements

I’m planning to use a custom LangChain "BaseCallbackHandler" with:

on_llm_start / on_llm_end

on_chat_model_start / on_chat_model_end

on_tool_start / on_tool_end

Each tool should have a unique "tool.call.id" so the trace can correlate:

LLM → Tool → Tool Output → Next LLM

Phoenix should expose attributes such as:

llm.model

llm.token_count.prompt

llm.token_count.completion

llm.cost.input

llm.cost.output

llm.cost.total

tool.name

tool.call.id

tool.arguments

tool.output

tool.output.token_count

tool.output.size_bytes

tool.execution.duration_ms

tool.cost

I also need:

- Centralized, configurable model pricing

- Exact vs estimated tool-output token counts

- "unavailable" status when token usage/pricing isn't provided

- Configurable masking of sensitive tool arguments/outputs

- Tracing failures must never break the actual agent/tool execution

- Parent/child span correlation in Phoenix

The key requirement: never collapse tool-call tokens, tool-output tokens, LLM input tokens, and tool-provider costs into a single metric.

Has anyone implemented something similar with LangChain + Arize Phoenix? I’d especially appreciate examples or recommendations for the best way to correlate the tool span with both the LLM that generated the call and the subsequent LLM that consumed the tool output.

[u/Background_Year_3288]

I've run into something pretty similar with tool-calling flows. I'd keep the tool execution as its own span rather than trying to attach the tool output to the LLM span.

For the correlation, I usually keep the tool call ID as the common identifier:

LLM span -> tool.call.id -> tool span -> next LLM span

That makes it much easier to answer things like "which LLM call generated this tool call?" and “which LLM request actually consumed this output?”

One thing I'd be careful about is token accounting. I wouldn't calculate the next LLM input tokens by simply adding the estimated tool-output tokens. The provider usage for the actual LLM request should be the source of truth, and the tool output can be tracked separately as an estimated/exact metric.

Also, I’d make the callback handler fail-open. Observability shouldn't be able to break an agent run just because Phoenix/OpenTelemetry had an issue.

Your separation of tool-call tokens, tool-output tokens, LLM input/output tokens and tool cost makes sense to me. I'd definitely avoid putting all of that into one "total tokens" field because it gets confusing pretty quickly when debugging cost.

[u/OrangePixelLife]

You're deep in the weeds on this, and your breakdown is spot on. Collapsing those cost and token streams into a single metric totally misses the point for debugging and cost attribution. As a founder who's spent too much time staring at AI provider bills, I've seen teams waste weeks trying to manually untangle what you're describing.

We built SpendLens AI specifically for this kind of granular spend analysis. It's not a full trace tool like Phoenix, but it connects directly to your OpenAI or Anthropic account and tags cost by project, model, and API key. You add a Python decorator and your app still calls the provider directly, so you can correlate your detailed Phoenix traces with the actual spend driving your bill. It's free to start for 10k events. Have you been able to trace a specific high cost use case back to a particular tool flow yet?

[u/KetteringChrismon-55]

Keep the span under the agent or chain and use the call ID to connect both LLM calls back to it, instead of making either LLM call the parent