Building evals that actually catch regressions — companion videos

Dave Ebbelaar

A working AI engineer walking through his actual eval ladder — assert-style unit tests, reference-free metrics, LLM-as-judge alignment with humans, and the analyze/measure/improve loop. The structure is the closest match on video to the article's argument that evals are a regression-catching system, not a leaderboard.

What you should get from this: Design an eval ladder that catches regressions before prompt or model changes reach users.

Watch or know first: Experience shipping or maintaining an AI workflow with known failure examples.

AI Engineer

Hamel Husain and Rechat's CTO walk through the eval system behind a real AI product: why generic off-the-shelf evals fail, a layered setup of assertions, logged traces with human review, and LLM judges kept aligned with a domain expert, and how a working eval system unlocks data curation and fine-tuning. It is the production case study for the article's argument that evals are regression-catching machinery, not a leaderboard.

What you should get from this: See how a production team layers assertions, human review and LLM judges so regressions surface before release.

Watch or know first: Comfortable with basic LLM evaluation terminology and the article's eval-ladder framing.