32:07Anyscale
Walks through why naive LLM serving wastes 60–80% of GPU memory, how PagedAttention borrows OS-style paging to fix that, and how continuous batching changes throughput. This provides background for the article's quality-equivalent cost model. Measure capacity and utilisation for your own workload instead of transferring a benchmark figure.
What you should get from this: Understand why serving engines, batching and KV-cache memory dominate self-hosted inference economics.
Watch or know first: Basic knowledge of transformer inference, GPU memory limits and API-vs-self-hosting tradeoffs.
AI Expert note: The PagedAttention mental model remains important, but throughput numbers and serving-engine defaults age quickly. Use this for fundamentals, then benchmark your own model, hardware, batch shape and uptime requirements before making a hosting decision.