Cost &amp; Model Operations

Using one model for everything is the rookie move. Production AI systems route different requests to different models — and save 60-90% on cost while improving quality. The patterns, the routing logic, and the trade-offs.

Multi-model orchestration: routing by cost, latency, and quality

Evaluate the implementation pattern, failure modes, and guardrails before building.

12 min read

LLM inference costs are 60-90% reducible with the right techniques. Prompt caching, model routing, output control, batching, and a few less-known patterns. The numbers, the patterns, and the production discipline that distinguishes well-run inference from a runaway bill.

Cost-optimizing inference: prompt caching, routing, and output control

Use the article as decision context for adoption, risk, governance, or investment choices.

11 min read

At what scale does self-hosting beat API calls? The actual math, the operational realities, and the patterns that distinguish teams who should self-host from teams who should keep paying for managed inference.

Self-hosted vs hosted inference: vLLM, TGI, and the break-even math

Use the article as decision context for adoption, risk, governance, or investment choices.

A working architect's view of the 2026 LLM stack — the model tiers, inference providers, orchestration layers, evaluation tooling, and the trade-offs that actually matter when shipping production AI. Everything you wish someone had laid out before you started.

The 2026 LLM stack: models, inference, tooling, and trade-offs

Use the article as decision context for adoption, risk, governance, or investment choices.

10 min read

Running AI locally has matured. With Ollama or LM Studio and a modern Mac, you can run capable models offline, free, and private. What works, what doesn't, and the use cases that actually benefit.

Local AI on your Mac: Ollama, LM Studio, and what 7B models can really do

Evaluate the implementation pattern, failure modes, and guardrails before building.

6 min read

A jargon-free comparison of free ChatGPT, ChatGPT Plus, and ChatGPT Pro — what changes when you upgrade and how to tell if you actually need to.

Free vs Paid ChatGPT: what you actually get for ~€20/month

Understand the idea well enough to try it safely in a low-risk setting.

New to AI

32 minutes

Anyscale. Walks through why naive LLM serving wastes 60–80% of GPU memory, how PagedAttention borrows OS-style paging to fix that, and why continuous batching produces the 24× throughput numbers the article uses in its math. After this, the article's "you'll be lucky to hit 50% utilisation" line stops feeling abstract.

Fast LLM Serving with vLLM and PagedAttention

56 minutes

OpenAI. OpenAI's own Build Hour on prompt caching — the 1024-token threshold, the prefix-stability requirement, audio caching at 99% discount for realtime, time-to-first-token impacts at long inputs. Useful when you are sizing the engineering effort to actually hit the cache reliably on your production prompts.

Build Hour: Prompt Caching

19 minutes

Prompt Engineering. Walks through Anthropic's prompt caching against Gemini's context caching with concrete latency-and-cost reductions per use case (long-document chat, few-shot, multi-turn). The breakdown of cache-write surcharge vs. cache-read discount is exactly what the article assumes when it talks about when caching pays off.

Is This the End of RAG? Anthropic's NEW Prompt Caching

40 minutes

Y Combinator. Karpathy's AI Startup School keynote frames LLMs as a new kind of computer — utility, fab, and OS rolled together — and argues for "partial autonomy" products with a human-controlled leash. It is the cleanest articulation of the stack-level mental model the article assumes: that you are picking inference vendors and tooling for a programmable substrate, not a chatbot.

Andrej Karpathy: Software Is Changing (Again)

211 minutes

Andrej Karpathy. This is the clearest end-to-end explanation on YouTube of what an LLM actually is — pretraining, tokenization, SFT, RLHF, reasoning RL, tool use, hallucinations — at the level of detail an engineer needs to reason about model trade-offs. Watch it once and the "GPT-class vs. open-weights vs. reasoning model" decisions in the article stop feeling like brand choices and start feeling like training-recipe choices.

Deep Dive into LLMs like ChatGPT

9 minutes

Matthew Berman. Walks through the LMSYS RouteLLM paper and code: a small classifier sits in front of a strong/weak model pair and decides which one to call, hitting roughly 95% of the strong model's quality at a fraction of the cost. The view count is under the usual 100k bar, but for the specific "show me real model routing, not just model comparisons" niche this is the cleanest explanation on YouTube and lines up directly with the article's quality/cost tradeoff section.

RouteLLM achieves 90% GPT4o Quality AND 80% CHEAPER

19 minutes

Tina Huang. A clean tour of the current model landscape grouped by tier — flagships, lite models, mid-tier, specialized — with concrete picks for what each tier is actually good for. This is the "know your options before you route" half of the article, and Huang frames cost-vs-capability the same way the article does without leaning on benchmark hype.

Every AI Model Explained