18 minutesAWS re:Invent 2025 - Implementing Human-in-the-Loop Controls for Multi-Agent AI Systems (CNS428)
See how approval gates can be implemented as explicit workflow checkpoints rather than informal manual review after something goes wrong.
Dave Ebbelaar. A working AI engineer walking through his actual eval ladder — assert-style unit tests, reference-free metrics, LLM-as-judge alignment with humans, and the analyze/measure/improve loop. The structure is the closest match on video to the article's argument that evals are a regression-catching system, not a leaderboard.
Some tool choices will age, but the ladder is sound: deterministic checks first, then model-graded checks validated against humans. Do not skip calibration just because an LLM judge is easy to add.
Design an eval ladder that catches regressions before prompt or model changes reach users.
Experience shipping or maintaining an AI workflow with known failure examples.
Last reviewed: May 18, 2026
Continue through the same learning path with the next curated companion videos.
18 minutesSee how approval gates can be implemented as explicit workflow checkpoints rather than informal manual review after something goes wrong.
4 minutesUnderstand why multilingual embeddings matter for private internal search and where local retrieval can reduce data-exposure risk.
17 minutesSet up a multi-agent coding workflow with explicit review boundaries so generated changes stay small, tested and owned.
Hand-picked external courses that go deeper on this topic.
University of Helsinki
The most trusted free AI introduction in Europe — built by the University of Helsinki, taken by over a million people, and available in English and 20+ other languages (including Estonian). No code, no math anxiety, and a certificate at the end. If you want the calm, credible version of 'what is AI actually', this is it.
Andrew Ng
Years after launch, still the cleanest starting point for anyone who needs to understand AI without learning to code. No math, no jargon, no hype — you'll finish able to have an informed conversation about AI projects.
Brenna Zenaty, Adriti Gulati
Closes our biggest vertical blind spot: nothing in the catalog spoke directly to support and customer-success teams. HubSpot Academy is free, well-produced, and refreshingly concrete — it walks through AI-assisted ticket triage and a knowledge-base agent rather than staying abstract, and the certificate is free too, not a paywall bait-and-switch.