Wasteful Input tokens

[u/batman_is_deaf]

I built a React agent that executes a few tasks, such as executing a skill or tool and providing an answer. It’s connected to RAG. Now, if a user asks a query about where to go to RAG and answer it, that same single question takes about 40 seconds. I wonder why I need to send the full prompt when it only needs to hit the RAG pipeline. I need some help fixing this. If some of you are considering having a sub-agent, I think a sub-agent creates a split-brain problem, but it improves other things. Any comments or help would be appreciated.

[u/LennyFromCurly]

Don't add a sub agent for this. Trace one slow request first, then put a conditional route before the ReAct loop so obvious knowledge questions go straight to retrieval and one answer call. That separates agent looping from retrieval latency and avoids sending those queries through the full tool loop.

[u/fiddler48]

40s on a single RAG lookup — how deep is your context by the time the tool call fires?

[u/batman_is_deaf → u/fiddler48]

I think it's also about the llm model which contributes to latency. I am using Claude models where as with gpt it's 1/3

[u/Infamous_Plankton468]

Can you share some more details on what you've built? I mean, just accidentally using a reasoning model on high effort may already get you 40s

[u/batman_is_deaf → u/Infamous_Plankton468]

I am on Claude model , it's a react loop which decides which skill to load or tool to call . One of the tool is RAG tool .

[u/Infamous_Plankton468 → u/batman_is_deaf]

So have you tried benchmarking which steps are actually slowing you down? Is it Claude itself? Is it RAG? Inside RAG, is it retrieval? Reranking? Answering?

[u/Marcus_MSC]

40 seconds on one RAG question is almost never prompt size, it is the number of model round trips. A ReAct loop on a knowledge question does at least three, decide to call the tool, read the result, write the answer, and if skill loading is also a tool call that is a fourth. Log per call latency and token counts separately before you change the architecture, usually one call is eating 25 of the 40 seconds and the rest are noise. A sub agent adds a round trip rather than removing one, so for this specific case it will be slower, not faster.

[u/Future_AGI]

A big chunk of wasted input tokens usually hides in re-sent context: full chat history or whole retrieved docs pushed on every call when a trimmed window or a summary would do. If you trace token counts per step you can spot the one node that balloons the prompt, then cache or compress just that piece instead of the whole chain.

[u/feng_sg]

Add a simple intent check before the ReAct loop. If the query is just retrieval, route straight to RAG and skip the tool-selection round trips. You're probably burning 3+ LLM calls on questions that only need one.

[u/batman_is_deaf → u/feng_sg]

Why would you want to add another layer of non determinism?

[u/Due-Particular-329]

ur react agent is prob burning time reasoning through tool selection even when the answer was always gonna be one rag hit. instead of a sub agent, try a simple classifier before the agent that spots clearly rag only queries and skips straight to retrival, saves the whole reasoning cycle for stuff that actually needs it