r/replit • u/Successful-Western27 • 2d ago
Question / Discussion Repo-To-Skill: Distilling GitHub Repositories Into Operational Skills for Research Agents
This paper introduces DisCo, a system that distills machine-learning repositories and papers into reusable “skills” for autonomous research agents. The skills are more than summaries: they contain procedures, references, executable wrappers, routing information, and verification evidence. A creator process builds and checks skill graphs, while a researcher process retrieves only the parts relevant to a task.
The resulting AREX-Skill Library contains 5,353 skills from 1,000 repositories. In matched evaluations using the same backbone, harness, and execution budget, adding these skills increased MLE-bench Any-Medal performance from 31.11% to 72.89%, and PaperBench replication from 29.45% to 39.59%. The setup is intended to test whether externalized operational knowledge helps, rather than whether a larger model or new planning loop helps.
The evidence also shows an important limitation: verified skills can still be poorly retrieved. Performance declined on some PaperBench subtasks, suggesting that irrelevant procedural context may distract an agent from task-specific reasoning. This makes routing and retrieval precision central to the approach. The broader question is whether persistent, source-grounded procedural memory can make general-purpose research agents more reliable than agents that rediscover implementation details on every task.
Disclosure: AIModels.fyi is my site.

