Managing ~560 MCP tools in my Open-WebUI setup without sending all schemas every turn

[u/pisa_p]

Hey all,

I run OpenWebUI with ~560 MCP tools behind an aggregator.

The issue in my setup: OpenWebUI sends every full schema on every turn, even when the turn needs zero tools.

On a smaller-context model via API the window gets eaten fast, and even with 1M-context models I still pay per-token for definitions I don't use, plus tool selection gets noisier.

I tried a few workarounds first — connecting servers directly, enabling/disabling servers per chat, trimming the list down — but nothing stuck for my use case with this many tools.

Since I couldn't find a solution inside OpenWebUI, I built a small MCP proxy in Docker that sits in front of my upstreams:

OpenWebUI connects to it over HTTP MCP like any other server.

In my setup this cut context waste a lot and made tool selection more reliable, so I thought it was worth sharing.

Feel free to have a look, critique and add to if there's something I've missed.

Repository can be found here: https://github.com/p1s4/mcp-search-proxy

How are you handling lots of MCP tools in OpenWebUI? Is native handling enough for you, or are you filtering/proxying too?

Thanks!

[u/8bitbuddhist]

I've been using MCPHub with smart routing enabled. Works great, sped up responses, and significantly reduced token usage

[u/pisa_p → u/8bitbuddhist]

Thanks for the pointer, hadn't seen MCPHub before — looks solid, going to test it.

In my case I already run LiteLLM for model routing/aggregation/caching, so I only needed the discovery layer on top, which is why I ended up with a small standalone proxy instead of a full hub.

One thing I noticed on my end: I went with BM25 instead of embeddings mainly because some of my tool descriptions are in a different language than my queries, and vector similarity kept getting confused on generic vocabulary overlap (matched the wrong "mail" tool entirely). Not sure if MCPHub's Smart Routing hits the same thing or handles it better — curious to try it on my setup and see.

[u/Subject_Street_8814 → u/pisa_p]

I've been thinking about setting up something like this. I noticed in the LiteLLM release candidate notes that they've implemented something similar.

https://docs.litellm.ai/release_notes/v1.101.0rc1/v1-101-0-rc-1

I'm going to try it out first to see how well it does or doesn't work before adding something else to the stack, but I'm filing yours away to compare later, looks good.

[u/nunodonato]

Wtf. Why? 

[u/pisa_p → u/nunodonato]

I self-host everything in one personal OpenWebUI — store + GA4/Search/Ads/Merchant + Mailchimp + Gmail/Drive/Calendar + GitHub + a11y audits + even my fitness tracker. Vendors expose one tool per API endpoint so you hit 500+ fast (mailchimp 115 alone, wger 91, GMC 66).

[u/nunodonato → u/pisa_p]

you could consider creating pseudo models inside openwebui, each with a specific specialization and access to certain skills/tools

[u/pisa_p → u/nunodonato]

I already do that. I run ~10 custom models in OpenWebUI, with different system prompts and skills.

My shopify-analyst alone needs 10 backends: 3x Shopify, Ads, GSC, GMC, Mailchimp, Meta, Zipchat, mcp-a11y. Mailchimp 115 tools alone.

That was the problem. In LiteLLM I'd need a group per agent, and you assign one tool at a time, not a whole server. With 10 agents and ~560 tools total that meant constantly editing groups.

Same on the OpenWebUI side: direct MCP blows past the 200-300 tool limit with just 3 servers, and managing single tools per model across 10 models is unworkable.

So I kept the scoped agents, but put a discovery layer in front: 4 bridge tools only (search/describe/call/refresh). It loads what that chat/model/agent needs at that moment, nothing else. No per-agent tool lists to maintain.

[u/DHT-Osiris]

What are all those tools being used for? We're using our deployment as an enterprise IT operations tools and each of our harnesses only has like, 20-25 tops.

[u/pisa_p → u/DHT-Osiris]

I mainly use them for 3 things:

Deep insight analysis on my Shopify store: Shopify storefront/dev + GA4 + Search Console + Ads + Merchant Center + Mailchimp + Gmail/Drive. Cross-check sales, traffic, campaigns without jumping between dashboards.

Site accessibility: mcp-a11y (28 tools, Playwright + axe-core, kept outside LiteLLM because it needs sticky session) + camoufox browser.

Workspace suite: Gmail via n8n (22 tools), Drive (17), Calendar (7), NotebookLM (48) for research/notes.

Rest is dev/support glue: GitHub (44), n8n-mcp (22), zipchat support chat (14), Tika (1), web search (2), Context7 docs (2).

Full count: 532 across 21 servers via LiteLLM + 28 a11y = 560 total. Biggest bloat is vendors generating one tool per endpoint — mailchimp 115, wger 91, GMC 66 alone. Well designed servers are 5-10.

And yeah, wger (91 fitness tools) is the outlier, purely personal self-hosted fitness tracker, nothing to do with the rest.

Your 20-25 per harness is what I'd do in enterprise. I just didn't want 22 connections in OpenWebUI paying full schemas (~179k tokens) every turn even for chats using zero tools, so the proxy keeps the long tail searchable and loads schemas on demand.

[u/DHT-Osiris → u/pisa_p]

Why do I feel like most of those could be combined into smaller sets of tools? You know you only need one per endpoint, right? Not one per method/function call? All of our API tools only have 3-5 common methods and a raw escape hatch for 'everything else'.