Topic
Production LLM Apps
Architect, ship, observe, and operate LLM applications after the demo stage.
31 stories (20 articles · 11 videos)
Start here
A few good first pieces before you browse the full feed.
13 min readArticle
The 2026 LLM stack: models, inference, tooling, and trade-offs
A working architect's view of the 2026 LLM stack — the model tiers, inference providers, orchestration layers, evaluation tooling, and the trade-offs that actually matter when shipping production AI. Everything you wish someone had laid out before you started.
Use the article as decision context for adoption, risk, governance, or investment choices.
Advanced
12 min readArticle
Designing prompts for production: system, developer, and user layers
Production prompts are not 'tell the AI what you want'. They are a layered system — stable instructions, dynamic context, per-call variables — managed like code. The architecture, the patterns, and the discipline that distinguishes production from prototype.
Separate system, developer, and user instructions and test production prompts as versioned system components.
Advanced
10 min readArticle
Production AI failure modes: what breaks after the demo
AI systems usually fail in predictable ways: hallucination, stale context, sycophancy, prompt injection, unsafe tool use, schema drift, and weak fallbacks. A production failure-mode register for teams shipping real workflows.
Build a production AI failure-mode register with controls for hallucination, stale context, prompt injection, unsafe tool use, and weak fallbacks.
Advanced
More in this topic
48 minutesVideo
How to Build Reliable AI Agents (Context + Evals Explained) | Tobias Leong, Axium
Arize AI. Explains why production agents fail when the system lacks the right context, evaluation data, tracing and domain expertise. It maps well to the article's failure-mode register because it makes reliability an engineering loop: separate retrieval from reasoning, define expected outcomes, evaluate tool calls, and trace failures before changing models.
Advanced
13 min readArticle
Shipping an LLM product: pricing, margins, and the anti-moat trap
LLM-powered products face economics that are harder than traditional SaaS. Variable costs that scale with usage, margins squeezed by inference, commoditization risk, and competitors with the same foundation models. How to build a product that's actually defensible — and the patterns that lead to LLM
Use the article as decision context for adoption, risk, governance, or investment choices.
Advanced
12 min readArticle
Cost-optimizing inference: prompt caching, routing, and output control
LLM inference costs are 60-90% reducible with the right techniques. Prompt caching, model routing, output control, batching, and a few less-known patterns. The numbers, the patterns, and the production discipline that distinguishes well-run inference from a runaway bill.
Use the article as decision context for adoption, risk, governance, or investment choices.
Advanced
14 min readArticle
Prompt injection and LLM security: threat models and defense-in-depth
Prompt injection is a permanent LLM security class, not a prompt-writing mistake. A production guide to threat models, data boundaries, tool permissions, regression tests, monitoring, and incident response.
Threat-model an LLM workflow and add concrete controls for untrusted content, retrieval, tool calls, authorization, monitoring, and incident response.
Advanced
12 min readArticle
Computer use and browser agents in production
Computer use and browser agents have demos that go viral. Production deployments at scale have a different shape — narrow scoping, heavy guardrails, careful UX. The patterns that work, the failures we keep seeing, and the honest economics.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
12 min readArticle
Building memory for long-running agents
Agents need memory beyond the context window. Long-term memory architecture — what to store, when to retrieve, how to forget — determines whether agents feel like they 'know' you or start fresh every conversation. The patterns and the production trade-offs.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
12 min readArticle
Context engineering: managing 1M-token windows without context rot
1M-token context windows exist, but quality degrades long before that limit. Context engineering is the discipline of using context windows effectively — what to include, what to summarize, what to retrieve fresh, and the patterns that keep quality high as context grows.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
11 min readArticle
LangGraph vs CrewAI vs direct API: choosing an agent framework in 2026
The agent framework landscape in 2026 is more mature but no clearer. LangGraph, CrewAI, Pydantic AI, OpenAI Agents SDK, and direct API — each fits some teams and projects, none fits all. A honest comparison and a decision framework.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
13 min readArticle
Designing agents that don't loop forever
The most common production agent failure is infinite or pseudo-infinite loops — agents that retry, branch, and burn through tokens without making progress. The architectural patterns that prevent this and produce agents that finish, even on hard tasks.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
13 min readArticle
Fine-tuning in 2026: when LoRA beats RAG, and how to do it without a cluster
LoRA fine-tuning has become accessible — you can run real fine-tunes on a laptop or rent a GPU for an hour. The patterns that work, the cases where fine-tuning beats RAG, and a practical end-to-end workflow from data prep to deployment.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
12 min readArticle
Choosing between prompting, RAG, and fine-tuning (and when to combine)
Prompting, RAG, and fine-tuning are the three big levers for adapting LLMs to your problem. Each is right for some problems and wrong for others. A framework for choosing, the realistic costs of each, and the production patterns where combining them shines.
Use the article as decision context for adoption, risk, governance, or investment choices.
Advanced
12 min readArticle
Building a production RAG: ingestion, embedding, retrieval, reranking, eval
A production RAG pipeline is six stages, each with specific patterns that determine quality. The architecture, the choices at each stage, and the iterative evaluation discipline that distinguishes RAG that works from RAG that disappoints.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
12 min readArticle
Designing MCP tools that LLMs actually use correctly
Most MCP tools we see are technically correct and practically useless. LLMs ignore them, misuse them, or call them in unhelpful ways. The principles for designing tools LLMs adopt naturally, with examples of common failures and their fixes.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
14 min readArticle
MCP from scratch: build a production-ready server in TypeScript
Building a production Model Context Protocol server requires more than wiring up a few tools. The patterns for schema design, auth, error handling, streaming, observability, and the production realities that make MCP servers useful at scale.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
12 min readArticle
Observability for LLM apps: tracing, costs, latency, quality drift
LLM applications fail in unique ways that traditional observability misses. The patterns for tracing multi-step flows, tracking costs that vary 100x per call, monitoring quality drift, and debugging hallucinations at production scale.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
13 min readArticle
Building evals that actually catch regressions
Most eval suites look impressive but miss real regressions. Building evals that catch what matters requires careful dataset construction, sensitive metrics, judge calibration, and a culture of trust. The patterns from teams that get this right.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
13 min readArticle
Structured outputs and function calling: the production patterns
Structured outputs and function calling are the bridge from 'LLM that generates text' to 'system that does work'. In production, the patterns that matter are about schemas, error handling, idempotency, and graceful degradation — not just JSON mode.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Advanced
10 min readArticle
Multi-model orchestration: routing by cost, latency, and quality
Using one model for everything is the rookie move. Production AI systems route different requests to different models — and save 60-90% on cost while improving quality. The patterns, the routing logic, and the trade-offs.
Evaluate the implementation pattern, failure modes, and guardrails before building.
Intermediate
42 minutesVideo
Vertical AI Agents Could Be 10X Bigger Than SaaS
Y Combinator. The Lightcone hosts work through why vertical AI agents — not horizontal wrappers — are the defensible shape for application-layer companies, with concrete examples and a clear-eyed take on which categories the model providers will eat. That is the anti-moat trap the article warns about, expressed as a positive playbook.
Advanced
34 minutesVideo
How AI is Reinventing Software Business Models ft. Bret Taylor of Sierra
Sequoia Capital. Bret Taylor walks through the shift from per-seat SaaS to outcomes-based pricing — what to anchor on (resolution, CSAT, NPS), why incumbents struggle to follow, and how vertical specialisation creates pricing power. It directly mirrors the article's pricing and margin sections.
Advanced
154 minutesVideo
Instrumenting & Evaluating LLMs
Hamel Husain. Hamel Husain, Eugene Yan, Brian Bischof, Harrison Chase, and Shreya Shankar working through tracing, log analysis, LLM-as-judge, and the workflow around looking at real production data. Sit with it the same way you would a long podcast — it is the single best deep treatment of the article's "look at your traces" thesis on YouTube.
Advanced
9 minutesVideo
LangSmith in 10 Minutes
LangChain. A guided tour of an LLM trace, project, and dataset by LangChain's co-founder — token cost, latency, error rate, feedback aggregation, drilling into a single retrieval-step span. It's the closest visual analogue to what the article describes when it talks about "every call is a span" and why structured traces beat print logging.
Advanced
25 minutesVideo
Prompting 101 | Code w/ Claude
Anthropic. A live build session by Anthropic's Applied AI team on an insurance-claims prompt — they start with a vague instruction and iterate to something a developer would actually ship, showing the kind of revisions the article describes for the system and developer layers. Watch this before re-reading the article's checklist on examples, output structure, and refusal handling.
Advanced
77 minutesVideo
AI prompt engineering: A deep dive
Anthropic. Four Anthropic prompt engineers (research, alignment, applied, developer relations) talking at length about what they actually do day to day — how they edit prompts under pressure, how they think about "honesty" in instructions, when XML scaffolds help, when they don't. The article's layered model maps cleanly onto how they describe the work; this is the best way to hear that mental model out loud.
Advanced
41 minutesVideo
OpenAI DevDay 2024 | Structured outputs for reliable applications
OpenAI. Walks through `strict: true`, the difference from old JSON mode, refusal handling, and how function calling and response-format schemas compose. Useful precisely because it describes the contract the API gives you, which is what the article's production patterns are built on top of.
Advanced
18 minutesVideo
Pydantic is all you need: Jason Liu
AI Engineer. The talk that crystallised the modern "define a Pydantic model, hand it to the LLM, let validation do the rest" pattern, with concrete examples of nested objects, validators that catch hallucinated URLs, and Chain-of-Thought as a typed field. Watch it before re-reading the article's section on validators and you will recognise where its retry and refusal rules come from.
Advanced
40 minutesVideo
Andrej Karpathy: Software Is Changing (Again)
Y Combinator. Karpathy's AI Startup School keynote frames LLMs as a new kind of computer — utility, fab, and OS rolled together — and argues for "partial autonomy" products with a human-controlled leash. It is the cleanest articulation of the stack-level mental model the article assumes: that you are picking inference vendors and tooling for a programmable substrate, not a chatbot.
Advanced
211 minutesVideo
Deep Dive into LLMs like ChatGPT
Andrej Karpathy. This is the clearest end-to-end explanation on YouTube of what an LLM actually is — pretraining, tokenization, SFT, RLHF, reasoning RL, tool use, hallucinations — at the level of detail an engineer needs to reason about model trade-offs. Watch it once and the "GPT-class vs. open-weights vs. reasoning model" decisions in the article stop feeling like brand choices and start feeling like training-recipe choices.
Advanced