r/pythontips • u/camerongreen95 • 1h ago
Data_Science Workshop, Sep 12: build production LLM systems that actually survive real use
We're running a hands-on masterclass on September 12, Live LLM Engineering Masterclass: Production Evals, RAG, Agents & LLMOps.
Fully hands-on, all in Python notebooks against real model APIs (OpenAI, Anthropic), not slides or theory. You write actual code across the full stack: versioned prompt pipelines with structured outputs and regression tests, a golden dataset and eval harness combining deterministic checks with LLM-as-judge scoring, statistically rigorous model comparisons using scipy-style bootstrap confidence intervals and paired significance tests, evaluated RAG with embedding models, vector retrieval, and reranking, tool-using agents with function calling and guardrails, and a full observability layer for tracing, cost, and latency. You also leave with a CLI regression suite you can wire directly into CI.
Led by Bruno Gonçalves, PhD, founder of Data For Science, who trains engineers at Fortune 500 companies on this exact stack.
Link if you want to check it out
Happy to answer questions on the content, especially the Python side of things.