Production LLM Apps

48 minutes

Video

How to Build Reliable AI Agents (Context + Evals Explained) | Tobias Leong, Axium

Arize AI. Explains why production agents fail when the system lacks the right context, evaluation data, tracing and domain expertise. It maps well to the article's failure-mode register because it makes reliability an engineering loop: separate retrieval from reasoning, define expected outcomes, evaluate tool calls, and trace failures before changing models.

Advanced

13 min read

Article

Shipping an LLM product: pricing, margins, and the anti-moat trap

LLM-powered products face economics that are harder than traditional SaaS. Variable costs that scale with usage, margins squeezed by inference, commoditization risk, and competitors with the same foundation models. How to build a product that's actually defensible — and the patterns that lead to LLM

Use the article as decision context for adoption, risk, governance, or investment choices.

Advanced

12 min read

Article

Cost-optimizing inference: prompt caching, routing, and output control

LLM inference costs are 60-90% reducible with the right techniques. Prompt caching, model routing, output control, batching, and a few less-known patterns. The numbers, the patterns, and the production discipline that distinguishes well-run inference from a runaway bill.

Use the article as decision context for adoption, risk, governance, or investment choices.

Advanced

14 min read

Article

Prompt injection and LLM security: threat models and defense-in-depth

Prompt injection is a permanent LLM security class, not a prompt-writing mistake. A production guide to threat models, data boundaries, tool permissions, regression tests, monitoring, and incident response.

Threat-model an LLM workflow and add concrete controls for untrusted content, retrieval, tool calls, authorization, monitoring, and incident response.

Advanced

12 min read

Article

Computer use and browser agents in production

Computer use and browser agents have demos that go viral. Production deployments at scale have a different shape — narrow scoping, heavy guardrails, careful UX. The patterns that work, the failures we keep seeing, and the honest economics.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

12 min read

Article

Building memory for long-running agents

Agents need memory beyond the context window. Long-term memory architecture — what to store, when to retrieve, how to forget — determines whether agents feel like they 'know' you or start fresh every conversation. The patterns and the production trade-offs.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

12 min read

Article

Context engineering: managing 1M-token windows without context rot

1M-token context windows exist, but quality degrades long before that limit. Context engineering is the discipline of using context windows effectively — what to include, what to summarize, what to retrieve fresh, and the patterns that keep quality high as context grows.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

11 min read

Article

LangGraph vs CrewAI vs direct API: choosing an agent framework in 2026

The agent framework landscape in 2026 is more mature but no clearer. LangGraph, CrewAI, Pydantic AI, OpenAI Agents SDK, and direct API — each fits some teams and projects, none fits all. A honest comparison and a decision framework.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

13 min read

Article

Designing agents that don't loop forever

The most common production agent failure is infinite or pseudo-infinite loops — agents that retry, branch, and burn through tokens without making progress. The architectural patterns that prevent this and produce agents that finish, even on hard tasks.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

13 min read

Article

Fine-tuning in 2026: when LoRA beats RAG, and how to do it without a cluster

LoRA fine-tuning has become accessible — you can run real fine-tunes on a laptop or rent a GPU for an hour. The patterns that work, the cases where fine-tuning beats RAG, and a practical end-to-end workflow from data prep to deployment.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

12 min read

Article

Choosing between prompting, RAG, and fine-tuning (and when to combine)

Prompting, RAG, and fine-tuning are the three big levers for adapting LLMs to your problem. Each is right for some problems and wrong for others. A framework for choosing, the realistic costs of each, and the production patterns where combining them shines.

Use the article as decision context for adoption, risk, governance, or investment choices.

Advanced

12 min read

Article

Building a production RAG: ingestion, embedding, retrieval, reranking, eval

A production RAG pipeline is six stages, each with specific patterns that determine quality. The architecture, the choices at each stage, and the iterative evaluation discipline that distinguishes RAG that works from RAG that disappoints.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

12 min read

Article

Designing MCP tools that LLMs actually use correctly

Most MCP tools we see are technically correct and practically useless. LLMs ignore them, misuse them, or call them in unhelpful ways. The principles for designing tools LLMs adopt naturally, with examples of common failures and their fixes.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

14 min read

Article

MCP from scratch: build a production-ready server in TypeScript

Building a production Model Context Protocol server requires more than wiring up a few tools. The patterns for schema design, auth, error handling, streaming, observability, and the production realities that make MCP servers useful at scale.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

12 min read

Article

Observability for LLM apps: tracing, costs, latency, quality drift

LLM applications fail in unique ways that traditional observability misses. The patterns for tracing multi-step flows, tracking costs that vary 100x per call, monitoring quality drift, and debugging hallucinations at production scale.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

13 min read

Article

Building evals that actually catch regressions

Most eval suites look impressive but miss real regressions. Building evals that catch what matters requires careful dataset construction, sensitive metrics, judge calibration, and a culture of trust. The patterns from teams that get this right.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

13 min read

Article

Structured outputs and function calling: the production patterns

Structured outputs and function calling are the bridge from 'LLM that generates text' to 'system that does work'. In production, the patterns that matter are about schemas, error handling, idempotency, and graceful degradation — not just JSON mode.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Advanced

10 min read

Article

Multi-model orchestration: routing by cost, latency, and quality

Using one model for everything is the rookie move. Production AI systems route different requests to different models — and save 60-90% on cost while improving quality. The patterns, the routing logic, and the trade-offs.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Intermediate

42 minutes

Video

Vertical AI Agents Could Be 10X Bigger Than SaaS

Y Combinator. The Lightcone hosts work through why vertical AI agents — not horizontal wrappers — are the defensible shape for application-layer companies, with concrete examples and a clear-eyed take on which categories the model providers will eat. That is the anti-moat trap the article warns about, expressed as a positive playbook.

Advanced

34 minutes

Video

How AI is Reinventing Software Business Models ft. Bret Taylor of Sierra

Sequoia Capital. Bret Taylor walks through the shift from per-seat SaaS to outcomes-based pricing — what to anchor on (resolution, CSAT, NPS), why incumbents struggle to follow, and how vertical specialisation creates pricing power. It directly mirrors the article's pricing and margin sections.

Advanced

154 minutes

Video

Instrumenting & Evaluating LLMs

Hamel Husain. Hamel Husain, Eugene Yan, Brian Bischof, Harrison Chase, and Shreya Shankar working through tracing, log analysis, LLM-as-judge, and the workflow around looking at real production data. Sit with it the same way you would a long podcast — it is the single best deep treatment of the article's "look at your traces" thesis on YouTube.

Advanced

9 minutes

Video

LangSmith in 10 Minutes

LangChain. A guided tour of an LLM trace, project, and dataset by LangChain's co-founder — token cost, latency, error rate, feedback aggregation, drilling into a single retrieval-step span. It's the closest visual analogue to what the article describes when it talks about "every call is a span" and why structured traces beat print logging.

Advanced

25 minutes

Video

Prompting 101 | Code w/ Claude

Anthropic. A live build session by Anthropic's Applied AI team on an insurance-claims prompt — they start with a vague instruction and iterate to something a developer would actually ship, showing the kind of revisions the article describes for the system and developer layers. Watch this before re-reading the article's checklist on examples, output structure, and refusal handling.

Advanced

77 minutes

Video

AI prompt engineering: A deep dive

Anthropic. Four Anthropic prompt engineers (research, alignment, applied, developer relations) talking at length about what they actually do day to day — how they edit prompts under pressure, how they think about "honesty" in instructions, when XML scaffolds help, when they don't. The article's layered model maps cleanly onto how they describe the work; this is the best way to hear that mental model out loud.

Advanced

41 minutes

Video

OpenAI DevDay 2024 | Structured outputs for reliable applications

OpenAI. Walks through `strict: true`, the difference from old JSON mode, refusal handling, and how function calling and response-format schemas compose. Useful precisely because it describes the contract the API gives you, which is what the article's production patterns are built on top of.

Advanced

18 minutes

Video

Pydantic is all you need: Jason Liu

AI Engineer. The talk that crystallised the modern "define a Pydantic model, hand it to the LLM, let validation do the rest" pattern, with concrete examples of nested objects, validators that catch hallucinated URLs, and Chain-of-Thought as a typed field. Watch it before re-reading the article's section on validators and you will recognise where its retry and refusal rules come from.

Advanced

40 minutes

Video

Andrej Karpathy: Software Is Changing (Again)

Y Combinator. Karpathy's AI Startup School keynote frames LLMs as a new kind of computer — utility, fab, and OS rolled together — and argues for "partial autonomy" products with a human-controlled leash. It is the cleanest articulation of the stack-level mental model the article assumes: that you are picking inference vendors and tooling for a programmable substrate, not a chatbot.

Advanced

211 minutes

Video

Deep Dive into LLMs like ChatGPT

Andrej Karpathy. This is the clearest end-to-end explanation on YouTube of what an LLM actually is — pretraining, tokenization, SFT, RLHF, reasoning RL, tool use, hallucinations — at the level of detail an engineer needs to reason about model trade-offs. Watch it once and the "GPT-class vs. open-weights vs. reasoning model" decisions in the article stop feeling like brand choices and start feeling like training-recipe choices.

Advanced

Start here

The 2026 LLM stack: models, inference, tooling, and trade-offs

Designing prompts for production: system, developer, and user layers

Production AI failure modes: what breaks after the demo

More in this topic

How to Build Reliable AI Agents (Context + Evals Explained) | Tobias Leong, Axium

Shipping an LLM product: pricing, margins, and the anti-moat trap

Cost-optimizing inference: prompt caching, routing, and output control

Prompt injection and LLM security: threat models and defense-in-depth

Computer use and browser agents in production

Building memory for long-running agents

Context engineering: managing 1M-token windows without context rot

LangGraph vs CrewAI vs direct API: choosing an agent framework in 2026

Designing agents that don't loop forever

Fine-tuning in 2026: when LoRA beats RAG, and how to do it without a cluster

Choosing between prompting, RAG, and fine-tuning (and when to combine)

Building a production RAG: ingestion, embedding, retrieval, reranking, eval

Designing MCP tools that LLMs actually use correctly

MCP from scratch: build a production-ready server in TypeScript

Observability for LLM apps: tracing, costs, latency, quality drift

Building evals that actually catch regressions

Structured outputs and function calling: the production patterns

Multi-model orchestration: routing by cost, latency, and quality

Vertical AI Agents Could Be 10X Bigger Than SaaS

How AI is Reinventing Software Business Models ft. Bret Taylor of Sierra

Instrumenting & Evaluating LLMs

LangSmith in 10 Minutes

Prompting 101 | Code w/ Claude

AI prompt engineering: A deep dive

OpenAI DevDay 2024 | Structured outputs for reliable applications

Pydantic is all you need: Jason Liu

Andrej Karpathy: Software Is Changing (Again)

Deep Dive into LLMs like ChatGPT