What the series is
A model is a stack of learned choices about numbers. This series walks those choices from the inside. The writing stays introductory, and it stays on the mechanism long enough that the name of a part matches a picture you can keep.
Each part stops at a result you can point at. A contextual vector. A controlled edit. A model trained on a preference. A forward pass that skips most of its own parameters. A model that answers on hardware you have.
The papers behind a claim sit at the bottom of the part that makes the claim. This page is the map between those parts.
How to read it
Read straight through on a first pass. A later part uses a word an earlier part defined, and it links back when it does. Part 1 is the vocabulary the other four spend.
Open the part that matches the work in front of you when you already know the block. An edit, a training run, a model that no longer fits, a reply that is too slow. The part stands on its own once those links make sense.
The order below is the order the ideas depend on each other. The homepage lists them the same way, with this guide first.
The five parts
-
1
The Core Engine Attention, SwiGLU, RoPE, and linear layers. A token becomes a vector that depends on its neighbors.
-
2
Visual Control Stable Diffusion, SAM, latent blending, and inpainting. Change a scene and keep a chosen object still.
-
3
How Models Learn Next-token prediction and denoising learn from raw data. RLHF and DPO learn from a preference.
-
4
Scaling with Experts Mixture of experts, sparse routing, and the gating network. Store more parameters than a pass runs.
-
5
Serving the Model The KV cache and grouped-query attention make decoding fast enough to serve. LoRA and QLoRA fine-tune on one GPU.
The thread
The same move shows up under different names. In part 1, attention selects tokens and SwiGLU selects features. In part 4, a gate selects whole feed-forward experts in that same slot. Three selections, one idea: most of what the model could use is left quiet.
Part 3 trains a diffusion model by learning to undo noise. The sampler in part 2 is that repair, started from a noisy latent and walked back toward an image. Editing a scene and training the model that paints it are the same loop, facing opposite directions.
Part 5 returns to the attention from part 1. The past keys and values are stored, so a new token is the only comparison the model has to run. The series opens on how a vector is built and closes on how to keep building it, one token at a time, on a machine that has to answer.
Where to start
Start with The Core Engine. When a later page surprises you, the surprise is one of the choices that page names: a neighbor weighted in, a feature gated out, an expert left idle, a past token read from cache. This guide is here when you want the map again.