AI Engineer. Hamel Husain and Rechat's CTO walk through the eval system behind a real AI product: why generic off-the-shelf evals fail, a layered setup of assertions, logged traces with human review, and LLM judges kept aligned with a domain expert, and how a working eval system unlocks data curation and fine-tuning. It is the production case study for the article's argument that evals are regression-catching machinery, not a leaderboard.
Recorded mid-2024; specific tools have moved on but the system design has not. Treat it as an architecture reference, not a vendor guide, and keep the domain expert in the loop as the talk insists.
See how a production team layers assertions, human review and LLM judges so regressions surface before release.
Comfortable with basic LLM evaluation terminology and the article's eval-ladder framing.
Last reviewed: Jul 15, 2026
Continue through the same learning path with the next curated companion videos.
18 minutesSee how approval gates can be implemented as explicit workflow checkpoints rather than informal manual review after something goes wrong.
4 minutesUnderstand why multilingual embeddings matter for private internal search and where local retrieval can reduce data-exposure risk.
17 minutesSet up a multi-agent coding workflow with explicit review boundaries so generated changes stay small, tested and owned.
Hand-picked external courses that go deeper on this topic.
University of Helsinki · MinnaLearnUniversity of Helsinki
The most trusted free AI introduction in Europe — built by the University of Helsinki, taken by over a million people, and available in English and 20+ other languages (including Estonian). No code, no math anxiety, and a certificate at the end. If you want the calm, credible version of 'what is AI actually', this is it.
Coursera · DeepLearning.AIAndrew Ng
Years after launch, still the cleanest starting point for anyone who needs to understand AI without learning to code. No math, no jargon, no hype — you'll finish able to have an informed conversation about AI projects.
HubSpot AcademyBrenna Zenaty, Adriti Gulati
Closes our biggest vertical blind spot: nothing in the catalog spoke directly to support and customer-success teams. HubSpot Academy is free, well-produced, and refreshingly concrete — it walks through AI-assisted ticket triage and a knowledge-base agent rather than staying abstract, and the certificate is free too, not a paywall bait-and-switch.
