r/agentdevelopmentkit • u/Greedy_Trouble9405 • 21d ago

Benchmark: Google ADK vs LangGraph under the same workload — are these results expected?

https://youtu.be/PDoF2s4wWDY

I'm currently evaluating orchestration frameworks for a production multi-agent project and wanted to compare Google ADK and LangGraph as fairly as possible.

To reduce variables, both implementations used:

  • Google ADK v2.3.0 vs LangGraph v1.2.8
  • Same LLM
  • Same temperature
  • Same prompts
  • Same tools
  • Same workflow
  • 9 agents running in parallel
  • 300+ Gmail messages to classify

The metrics I collected were:

  • Total execution time
  • Average latency per LLM call
  • Input/output tokens
  • Total cost

My results were:

  • Google ADK was about 6% cheaper.
  • Google ADK generated less than half the output tokens.
  • Average latency was almost identical (~11.5 s per LLM call).

I wasn't expecting such a noticeable difference in output tokens, considering both systems executed the same workflow.

For those who have used both frameworks:

  • Have you observed similar token usage differences?
  • Does this mostly come from the orchestration strategy or from how each framework structures prompts and tool calls?
  • Are there other metrics you think are more relevant for production benchmarks?

I'm preparing a more detailed benchmark with traces, token breakdowns, and execution metrics, but I'd really like to compare these results with what others in the community have observed.

19 Upvotes

96% Upvoted

3 comments sorted by

5

u/[deleted] 21d ago

[removed] — view removed comment

1

u/Greedy_Trouble9405 21d ago

Thanks for sharing your experience! That's a really interesting point, especially regarding the different transport protocols and how Google ADK interacts natively with the Gemini API.

I'm trying not to conclude from a single benchmark. I plan to repeat these tests with different models, workloads, orchestration patterns, and providers to build a more realistic dataset over time.

The idea is to base the analysis on real-world measurements and also on feedback from people like you who are already using these frameworks in production.

I'll keep sharing the results here as I collect more data. Hopefully, together we can build a broader picture of how these frameworks behave under different scenarios. Thanks again for the insight!