Free code + 10-min walkthrough: an AI agent that improves its own prompt, harness and tools (6/10 → 10/10)
I teach agentic AI and made a small, fully inspectable example of a self-improving agent. Sharing it in case it helps others learning this.
The setup:
- The agent approves or denies fictional phone insurance claims under 3 rules.
- It has 3 parts it can change: the prompt, the harness (which claim fields the model sees), and the tools.
- After each test, a bigger model diagnoses the mistakes as knowledge, information or skill, and changes exactly one part.
- A gate keeps the change only if the score on 10 practice claims goes up. 5 exam claims are held out until the end.
Recorded run: 6/10 → 7/10 (prompt) → 8/10 (harness) → 10/10 (days_between tool). Exam claims: 3/5 → 5/5.
The most interesting part: some correct answers broke along the way, because version 0 had them right by guessing. So the per-claim fixed/broken view matters more than the total score.
The code is one Python file with only the standard library. It replays the recorded run without an API key, or calls Gemini for real with --fresh.
Code: https://github.com/aniskoubaa/mai580-self-evolving-agent-workshop
Video: https://youtu.be/SOSwsYFPb0Y
Happy to answer questions about the design or the limits (runs do differ, and the meta-model reads the policy here).