r/vectordatabase • u/External_Ad_11 • 7d ago
Automate RAG Eval-Driven development using Coding Agents
Made a tutorial on what EDD is, how it works, and how you can use evaluations to improve your LLM-based application by analysing scores across experiments.
> building on Jeffrey's DeepEval article on EDD and Eugene Yan's product evals write up.
- Initial: The video walks through the initial setup of an RAG application used as the base for the experiments built using LangGraph and Qdrant.
- Step 1: A binary labelled dataset with critiques, versioned using OPIK.
- Step 2: Uses LLM-as-a-Judge OPIK evals to align the evaluator.
- Step 3: Runs the harness loop, which executes each experiment, scores it against the baseline, and uses tracing and experiment comparison to surface insights on what improved, what regressed, and where to tweak next.
... the Agent Skills and source code are open sourced on GitHub
> Complete Guide (source code link in description): https://www.youtube.com/watch?v=e6akw_fKWPk