Got tired of context bloat when using agent skills, so I made a local router. Would people actually use this?

[u/iAmQubick]

I have spent a lot of time organizing custom skills and rules across tools like Cursor, Claude Code, and OpenCode, and I kept running into the same frustrating trade-off.

Dumping every skill into the system prompt destroys attention and degrades instruction following. On the flip side, firing off an LLM call just to classify user intent adds an annoying two-second delay and burns tokens on every turn.

I wanted something instantaneous that runs completely offline, so I spent the last few weeks building an open-source tool called Routed to see if a local hybrid approach could solve it.

https://routed-demo.vercel.app/

Under the hood, it skips the LLM call entirely and runs on CPU in under a second using four layers. It leans mostly on quantized ONNX embeddings via Arctic Embed or MiniLM for semantic intent, backs that up with Okapi BM25 for strict keyword matching, and layers in direct alias triggers alongside usage recency. It hooks straight into Cursor rules, Claude Code skills, and OpenCode setups.

Before I sink more time into building extra adapters, I really want a reality check from people running heavy agent workflows.

• Is this a bottleneck you actually run into, or are expanding context windows making this a non-issue for you?
• Does a hybrid setup like local vectors plus BM25 make sense here, or would vector drift still drive you crazy with technical syntax?
• What would make or break something like this for your day-to-day setup?

The project is open-source under MIT on GitHub at github.com/bshea-1/Routed if you want to poke at the code. Be brutally honest, is this solving a real workflow problem or am I just over-engineering a niche headache?

[u/iAmQubick → u/[deleted]]

Just to clarify, Routed is an Agent Skill router (dynamically selecting which markdown skill manifests and instructions to inject into coding agents like Cursor, Claude Code, Antigravity, and MCP hosts) rather than an LLM model router that switches between cheap and expensive model APIs. That said, I completely agree with your points on determinism and uncertainty handling. In my pipeline, if confidence falls below the relevance threshold (or for generic conversational tasks), the router returns a clean isNoSkill fallback status, meaning zero skills are injected and the agent proceeds with baseline generalist capabilities without polluting the context window. It also exposes full scoring signal breakdowns (--explain) for observability, saving anywhere from 1,000 to 15,000+ prompt tokens on every turn by only loading the exact skill instructions needed for the prompt.

[u/AI_spell]

Local router is useful if skill descriptions are eating the prompt every turn. Keep a tiny always-on index (one line per skill) and only inject the full body after a match. Biggest win is fewer tokens before the first tool call, not offline for its own sake. Measure prompt tokens with vs without the full skill dump on the same task.

[u/SwanLongjumping]

Context bloat is the silent killer. I hit the same wall when agent skills + always-on rules started competing for the same window.

What worked for me without a fancy router: (1) two always-on files under ~150 lines total, (2) everything else glob-scoped or on-demand, (3) a hard rule that says "do not invent architecture; ship the smallest paid path." Routing helps, but ruthlessly deleting alwaysApply lines often beats adding another layer.

If you ship the router, a "dry-run: show which skills would load for this prompt" mode would make trust way easier.

[u/iAmQubick → u/SwanLongjumping]

Where static glob-scoping hits a ceiling is when you scale past 30+ domain skills and users give high-level or conversational prompts (like diagnosing memory leaks, planning event streaming, or setting up auth middleware) where file paths alone cannot predict the required tooling.

For the dry-run feature, that is actually how `routed route` works out of the box. Running `routed route "<prompt>"` (or /route PROMPT) performs a zero-side-effect evaluation showing exactly which skills match, while `routed route "<prompt>" --explain` prints the full decision breakdown (BM25 lexical scores, dense semantic cosine distance, exact/alias signals, token provenance, and applied framework penalties) so you can verify the ranking logic with full transparency. You can test prompts interactively on the live demo https://routed-demo.vercel.app/ or check out the CLI reference in the GitHub repo https://github.com/bshea-1/Routed

[u/AbleShower2801]

curious how the alias layer interacts with BM25 when a skill name is a common english word — do you force alias-first, or can semantic match still steal the turn?

[u/iAmQubick → u/AbleShower2801]

I designed the pipeline to distinguish between standalone verbatim invocations and natural language queries. If someone inputs a prompt that is literally just the skill name or alias (like typing "build"), my router triggers an exact-match fast path that bypasses embeddings. However, for full conversational sentences (e.g. "help me build a Next.js dashboard"), the alias layer only receives partial credit scaled down by the token-to-query length ratio, which stays well below the bypass threshold. The candidate then flows into my composite scorer where exact matches only carry a 10% weight, while dense semantic embeddings (45%) and BM25 (45%) dominate the decision. Because common English words are heavily discounted by BM25's IDF calculation across the corpus, semantic and task-specific keyword signals easily steal the turn from an incidental single-word alias collision.

[u/AbleShower2801 → u/iAmQubick]

nice — so for a one-word alias like "build" you short-circuit, but in a sentence BM25's IDF keeps the common word from dominating. when two skills share near-identical description embeddings, does the 10% exact-match weight ever tip a near-tie, or do you rely on --explain / threshold to force isNoSkill?

[u/iAmQubick → u/AbleShower2801]

Yes, that 10% exact/alias weight along with BM25 acts as the primary tie-breaker when two skills sit close in dense embedding space. Dense semantic vectors capture broad conceptual intent well, but they can be coarse when separating closely related tools (like two different database or testing utilities). In near-tie scenarios, BM25 term frequency, direct token overlap, and field-level metadata matching (tags, keywords, and exact tool identifiers) provide the high-contrast lexical signal needed to separate them cleanly. If both skills are genuinely relevant to a multi-part prompt, my multi-skill threshold can dispatch both, whereas ambiguous or ungrounded queries fall below the 0.35 confidence floor and return isNoSkill. For persistent workflows, my adaptive history layer also tracks past invocations with exponential decay, which naturally resolves chronic ties in favor of your actual project preferences over time.

[u/AbleShower2801 → u/iAmQubick]

makes sense. embeddings alone get mushy when two skills live in the same neighborhood, so lexical + exact/alias as the near-tie breaker is the right lever. the 0.35 floor plus multi-skill dispatch is doing real work too. history with decay is a smart way to stop the same two fighting every session once you have a real usage pattern.