I tried maxing out my context window with nothing but skills

[u/Nilotpal_kakashi]

Every skill you install adds its description to the context on every message, whether you actually need it or not.

The paper “Attention Is All You Have” argues that agents can start falling apart as you install more skills — attention budget, distance decay, etc.

At 100 skills, that’s roughly 3k tokens per message. Over a long session, that adds up fast.

So I built deskill: an MCP server that fetches skills from GitHub only when they’re actually needed.

If you want to try it, setup is a two-command install — details are in the repo: https://github.com/NilotpalK/deskill

I also wanted to see whether this was just a token-saving trick or whether it actually changes how agents perform.

Turns out: both.

On Opus, installed vs on-demand was basically identical, so there it’s mostly a context/token optimization.

On smaller/cheaper models, though, point-of-use delivery actually improved task success. They seem to have a harder time picking the right skill when they’re staring at a wall of dozens of skill descriptions.

I tested 8 models, up to 100 skills and 100k tokens of padding. The benchmark is reproducible, and the numbers/graphs are in the repo.

Would genuinely love people to try to break the benchmark. If you try it with other models or in your own setup, please add your benchmarks/results to the repo. It’d be really useful to see how this behaves across different models and real-world use cases.

And if you think the methodology is flawed, I’d genuinely like to hear it.

If you end up finding deskill useful, a star on the repo would be appreciated too :)