Cost & Model Operations

13 min read

Article

The 2026 LLM stack: models, inference, tooling, and trade-offs

A working architect's view of the 2026 LLM stack — the model tiers, inference providers, orchestration layers, evaluation tooling, and the trade-offs that actually matter when shipping production AI. Everything you wish someone had laid out before you started.

Use the article as decision context for adoption, risk, governance, or investment choices.

Advanced

10 min read

Article

Local AI on your Mac: Ollama, LM Studio, and what 7B models can really do

Running AI locally has matured. With Ollama or LM Studio and a modern Mac, you can run capable models offline, free, and private. What works, what doesn't, and the use cases that actually benefit.

Evaluate the implementation pattern, failure modes, and guardrails before building.

Intermediate

6 min read

Article

Free vs Paid ChatGPT: what you actually get for ~€20/month

A jargon-free comparison of free ChatGPT, ChatGPT Plus, and ChatGPT Pro — what changes when you upgrade and how to tell if you actually need to.

Understand the idea well enough to try it safely in a low-risk setting.

New to AI

32 minutes

Video

Fast LLM Serving with vLLM and PagedAttention

Anyscale. Walks through why naive LLM serving wastes 60–80% of GPU memory, how PagedAttention borrows OS-style paging to fix that, and why continuous batching produces the 24× throughput numbers the article uses in its math. After this, the article's "you'll be lucky to hit 50% utilisation" line stops feeling abstract.

Advanced

56 minutes

Video

Build Hour: Prompt Caching

OpenAI. OpenAI's own Build Hour on prompt caching — the 1024-token threshold, the prefix-stability requirement, audio caching at 99% discount for realtime, time-to-first-token impacts at long inputs. Useful when you are sizing the engineering effort to actually hit the cache reliably on your production prompts.

Advanced

19 minutes

Video

Is This the End of RAG? Anthropic's NEW Prompt Caching

Prompt Engineering. Walks through Anthropic's prompt caching against Gemini's context caching with concrete latency-and-cost reductions per use case (long-document chat, few-shot, multi-turn). The breakdown of cache-write surcharge vs. cache-read discount is exactly what the article assumes when it talks about when caching pays off.

Advanced

40 minutes

Video

Andrej Karpathy: Software Is Changing (Again)

Y Combinator. Karpathy's AI Startup School keynote frames LLMs as a new kind of computer — utility, fab, and OS rolled together — and argues for "partial autonomy" products with a human-controlled leash. It is the cleanest articulation of the stack-level mental model the article assumes: that you are picking inference vendors and tooling for a programmable substrate, not a chatbot.

Advanced

211 minutes

Video

Deep Dive into LLMs like ChatGPT

Andrej Karpathy. This is the clearest end-to-end explanation on YouTube of what an LLM actually is — pretraining, tokenization, SFT, RLHF, reasoning RL, tool use, hallucinations — at the level of detail an engineer needs to reason about model trade-offs. Watch it once and the "GPT-class vs. open-weights vs. reasoning model" decisions in the article stop feeling like brand choices and start feeling like training-recipe choices.

Advanced

9 minutes

Video

RouteLLM achieves 90% GPT4o Quality AND 80% CHEAPER

Matthew Berman. Walks through the LMSYS RouteLLM paper and code: a small classifier sits in front of a strong/weak model pair and decides which one to call, hitting roughly 95% of the strong model's quality at a fraction of the cost. The view count is under the usual 100k bar, but for the specific "show me real model routing, not just model comparisons" niche this is the cleanest explanation on YouTube and lines up directly with the article's quality/cost tradeoff section.

Intermediate

19 minutes

Video

Every AI Model Explained

Tina Huang. A clean tour of the current model landscape grouped by tier — flagships, lite models, mid-tier, specialized — with concrete picks for what each tier is actually good for. This is the "know your options before you route" half of the article, and Huang frames cost-vs-capability the same way the article does without leaning on benchmark hype.

Intermediate

Start here

Multi-model orchestration: routing by cost, latency, and quality

Cost-optimizing inference: prompt caching, routing, and output control

Self-hosted vs hosted inference: vLLM, TGI, and the break-even math

More in this topic

The 2026 LLM stack: models, inference, tooling, and trade-offs

Local AI on your Mac: Ollama, LM Studio, and what 7B models can really do

Free vs Paid ChatGPT: what you actually get for ~€20/month

Fast LLM Serving with vLLM and PagedAttention

Build Hour: Prompt Caching

Is This the End of RAG? Anthropic's NEW Prompt Caching

Andrej Karpathy: Software Is Changing (Again)

Deep Dive into LLMs like ChatGPT

RouteLLM achieves 90% GPT4o Quality AND 80% CHEAPER

Every AI Model Explained