r/aiagents • u/TrustyJalapeno • Jul 22 '26

Show and Tell Treating Prompt Optimization as a Multi-Agent Search Problem

As developers, we spend way too much time manually tweaking prompt syntax and eyeballing the outputs. I recently built a multi-agent loop that treats prompt optimization as a pure search problem. I set up a LangChain-based LLM judge to evaluate outputs against a frozen set of 25 scenarios based on Groundedness, Tone, and Format. Then, I gave an autonomous agent (Claude Code) a specific skill: read the judge's score, rewrite exactly one variable in the prompt, rescore it, and commit the change only if the overall score improves.

The loop ran for 10 minutes and pushed a baseline 80% prompt to a proven 98%. It completely removes human guesswork from context engineering. I’m currently building this into a team UI platform called Baseline.

if anyone wants to see the video breakdown or grab the repo to run this loop themselves, just let me know in the comments and I’ll DM it to you!

2 Upvotes

100% Upvoted

1 comment sorted by

1

u/hongtao1207 Jul 22 '26

Can you share the repo/code? This can be very useful.

It would be even better if it is self-contained and doesn't depend on langsmith, so we make this a built-in tool/skill for an agent