Everything else — the conversation, the code, the apology when it gets something wrong — is engineering built on top of that one operation, repeated a few hundred times a second.
This page takes that sentence apart. What a token is, how meaning gets turned into numbers, what attention computes, what training actually changes, and why a system this capable still confidently invents things.
Three of the sections are instruments rather than prose. Type in them, drag them, break them. The concepts are exact; where a demo uses stand-in numbers instead of a live model, it says so.
Before anything else happens, your text is chopped into pieces from a fixed vocabulary. Almost every strange behaviour you have noticed starts here.
A model has a vocabulary — typically somewhere between 32,000 and 200,000 entries — and text must be expressed entirely in those entries. Common words get one token each. Rarer words get broken into fragments. The algorithm is usually byte-pair encoding: start from raw bytes, then repeatedly merge whichever adjacent pair occurs most often in the training corpus, until you have filled the vocabulary. Frequency buys wholeness.
Leading spaces belong to the token, which is why the and the are
different entries. As a rough English rule: one token ≈ 4 characters ≈ 0.75 words.
Faithful reproduction of GPT-2/3-style pre-tokenisation (the regex that splits contractions, leading spaces, digit runs and punctuation), followed by a simplified merge step standing in for the real learned merge table. Segment boundaries and counts are close to a production tokeniser, not identical to it.
1234567 may arrive as several digit-group
tokens, so column alignment is something the model reconstructs rather than perceives.Token 3,290 means nothing. The model's first move is to swap that ID for a long list of numbers — and those numbers are where meaning lives.
The embedding matrix is a lookup table with one row per vocabulary entry. Each row is a vector of a few thousand numbers. Nothing about it is hand-designed; the values are learned during training, adjusted by the same gradient descent that shapes everything else.
What emerges is geometry. Tokens used in similar contexts end up pointing in similar directions, and direction can carry relationships. The famous early demonstration came from word2vec in 2013: take the vector for king, subtract man, add woman, and the nearest neighbour is often queen. Relationships like gender, plurality and capital-of behave, loosely, like consistent offsets you can add.
One more thing matters, and it is the upgrade the transformer brought. A raw embedding is static — one vector per token, identical everywhere. As the vector passes up through the model's layers it becomes contextual: the representation of "bank" in river bank genuinely diverges from "bank" in bank transfer. The mechanism that does the diverging is the subject of the next part.
Each position asks a question, every earlier position answers, and the answers get averaged in proportion to how well they match.
From each token's vector the model projects three new vectors: a query (what I am looking for), a key (what I offer), and a value (what I will hand over if chosen). To update a position, take the dot product of its query against every key, turn those scores into percentages with a softmax, and mix the values by those percentages.
Two structural details do a lot of work. A causal mask sets every future position's score to −∞ before the softmax, so a token can only ever look backwards — that is what makes the model a valid next-token predictor. And attention runs in parallel heads, each with its own projections, so one head can track subject–verb agreement while another tracks quotation marks.
Weights here are illustrative, hand-set to show patterns that interpretability research does find in real heads — long-range subject tracking, pronoun and relative-clause binding. A live model's heads are messier and mostly uninterpretable at a glance.
Attention's cost is the reason long context is expensive: comparing every position against every other is quadratic in sequence length. Double the context, quadruple the attention work. Most of the last few years of systems research — FlashAttention, sliding windows, grouped-query attention, KV caching — is about paying that bill more cheaply.
Here is the entire journey from your text to a single next token. Then it happens again.
Two things worth holding onto. First, the residual stream — each block adds its output back into a running total rather than replacing it, so information can skip layers untouched. This is what makes networks hundreds of layers deep trainable at all.
Second, every token costs the same compute. The model cannot think harder about a difficult token; it can only produce more tokens. That single fact explains chain-of-thought prompting: asking for step-by-step working is not a psychological trick, it literally buys more forward passes to compute in. Reasoning models are trained with reinforcement learning to do this well, and to keep doing it until the answer stabilises.
A base model and a chat assistant are the same network at different points in the same process. Knowing which stage does what tells you which problems are fixable.
Feed the model an enormous amount of text and ask it, over and over, to predict the next token. Compare its distribution to the token that actually came next, measure the gap with cross-entropy loss, and nudge every weight to make the truth slightly more likely. No labels are needed — the text labels itself, which is why this scales to trillions of tokens. This stage is typically upwards of 98% of the total compute, and it is where essentially all factual knowledge and language ability is acquired.
The output is a base model: fluent, knowledgeable, and useless as an assistant. Ask it a question and it may well continue with more questions, because that is what a page of questions usually does.
Now train on a much smaller, curated set of written demonstrations: prompt, then an ideal response. Thousands to hundreds of thousands of examples, not trillions. The model learns the format of being helpful — answer the question, stop when done, hold a turn in a conversation. It learns almost no new facts here.
Demonstrations cannot express "this reply is a bit too smug." So instead: show humans two candidate responses, ask which they prefer, and use those comparisons as the signal. In classic RLHF the comparisons train a reward model, and the language model is then optimised against it with reinforcement learning. DPO and its relatives skip the separate reward model and optimise on the preference pairs directly. Anthropic's Constitutional AI variant has the model critique and revise its own outputs against an explicit written set of principles, so a large share of the preference labels are generated by AI rather than by people.
This is the stage that produces tone, refusals, formatting habits, and hedging. It is also where sycophancy comes from: if raters reliably prefer agreeable answers, agreeableness is exactly what gets optimised.
The network's output is not a word. It is a probability for every word. Something then has to choose — and how it chooses is a dial you control.
The final layer emits one raw score, a logit, per vocabulary entry. Softmax turns those into probabilities summing to 1. Temperature divides the logits before the softmax: below 1 it sharpens the distribution toward the favourite, above 1 it flattens it toward the field. Top-p (nucleus sampling) then keeps only the smallest set of candidates whose probabilities add up to p, and discards the long tail entirely.
Prompt type: . The same two dials behave completely differently across these.
Notice what the extremes do. Near zero, the model always picks its favourite — repeatable, and prone to loops and flat prose. Above roughly 1.3 the tail gets real probability mass, and text drifts from creative to incoherent. Top-p is the safety rail: it lets you raise temperature for variety while still refusing to ever sample genuine nonsense.
Then switch prompts, because this is the part that gets missed. On the factual prompt the dials barely matter — the evidence for Paris is so overwhelming that you have to push temperature past 1.5 before anything else gets a serious look, and top-p does nothing until it is almost at 1. On the open-ended prompt the same dials transform the output, because ten continuations were already close to tied. Temperature does not add creativity; it decides how much to respect a distribution the model has already committed to. Where that distribution is peaked, turning the dial mostly buys you errors rather than variety.
The decision to build very large models was not a hunch. It came from plotting loss against compute and finding a straight line on a log scale.
In 2020 Kaplan and colleagues showed that test loss falls as a smooth power law in model size, dataset size and compute — remarkably predictable across many orders of magnitude. That predictability is what made spending nine figures on a training run a defensible engineering decision rather than a gamble.
In 2022 the Chinchilla paper corrected the recipe. Given a fixed compute budget, the earlier generation had made models too big and trained them on too little data. Compute-optimal training, they found, scales parameters and tokens together — roughly 20 training tokens per parameter. The proof was direct: Chinchilla at 70B parameters outperformed Gopher at 280B, using the same compute budget spent differently.
| Model | Params | Training tokens | Tokens / param | What it showed |
|---|---|---|---|---|
| GPT-3 (2020) | 175B | ~300B | 1.7 | Scale alone produced few-shot learning |
| Gopher (2021) | 280B | ~300B | 1.1 | Bigger, similarly under-trained on data |
| Chinchilla (2022) | 70B | 1.4T | 20 | Beat Gopher at a quarter the size |
| Llama 3 8B (2024) | 8B | 15T | ~1875 | Deliberately trained far past optimal |
That last row is the part people miss. Chinchilla optimises training cost. If a model will serve billions of requests, inference cost dominates the budget, and it is worth over-training a small model well past the compute-optimal point to get a permanently cheaper one. Modern open models are trained on hundreds of times more data per parameter than the 2022 rule recommends — not because the rule was wrong, but because it answered a different question.
A second lever changes the arithmetic again: mixture of experts. Route each token to a small subset of many parallel feed-forward blocks, and total parameters can be far larger than the parameters actually used per token — capacity without proportional cost.
Each of these follows directly from something described above. That is what makes them predictable — and what tells you which ones tooling can fix.
The training objective rewards plausible, not true. There is no fact table inside the weights to consult and no internal signal that says "this next part is a guess" — a fabricated citation is generated by exactly the same machinery, at similar confidence, as a correct one. Fluency and accuracy are separate axes, and the model is optimised hard on one of them. Worse, saying "I don't know" is only produced if preference training rewarded it, which requires raters who can tell honest uncertainty from unhelpful hedging.
What helps: retrieval, so the answer is grounded in supplied text rather than recalled; tool use for anything computable; asking for sources you can check; and treating any specific number, name or citation as unverified by default.
The weights are frozen after training. The context window is working memory, and it is the only memory — start a new conversation and everything is gone. Products that appear to remember you are re-inserting stored notes into the prompt behind the scenes. Nothing you say is learned in the moment.
Show a model three examples of a format it has never seen and it will follow the pattern — with no weight update whatsoever. The pattern-matching happens inside a single forward pass, in the activations. This is why few-shot prompting works, and why its effects vanish the moment the conversation ends.
Base models are reasonably well calibrated — their probabilities roughly track how often they are right. Preference training tends to degrade that: optimising for answers humans like pushes toward confident phrasing regardless of underlying uncertainty. Assertive tone carries almost no information about reliability.
A model reading a web page, an email or a document cannot cryptographically distinguish your instructions from text that merely looks like instructions. That is the root of prompt injection, and it is a structural consequence of the fact that there is only one stream of tokens. Mitigations reduce it; nothing yet eliminates it.
Each of these is a reasonable inference from watching a model behave. Each is wrong in a way that changes how you should use one.
Short answers to the things that bring most people here. Each one links to the part of the page that works through it properly.
It converts your text into tokens, turns each token into a vector, passes those vectors through dozens of transformer layers that let every position read from earlier positions, and emits a probability for every entry in its vocabulary. One token is sampled from that distribution, appended to the input, and the whole process repeats. See the full trace →
A token is a chunk of text from a fixed vocabulary — usually a common word, a fragment of a rarer word, or punctuation. Models never see individual letters, only these chunks. In English one token averages about four characters, or roughly ¾ of a word. Try the segmenter →
Because the training objective rewards plausible text, not true text. There is no fact database inside to consult, and no internal signal marking a claim as a guess — a fabricated citation is produced by the same machinery, at similar confidence, as a correct one. Retrieval, tool use and checkable sources help; the tendency is structural, not a bug to be patched. Read why →
Temperature divides the model's raw scores before they become probabilities. Below 1 it sharpens the distribution toward the single most likely token; above 1 it flattens it so unlikely tokens get a real chance. It does not add creativity — it decides how strictly to respect a distribution the model already computed. Drag the dial →
A weighted lookup. Each position emits a query, every position offers a key and a value; the dot product of query against key becomes a percentage after softmax, and the values are mixed in those proportions. A causal mask stops any position reading the future. See the pattern →
No. The weights are frozen after training, and the context window is the only memory — it is cleared when the conversation ends. Products that appear to remember you are storing notes separately and re-inserting them into the prompt. More on this →
Use retrieval for knowledge — anything that changes, needs citing, or must be current. Use fine-tuning for behaviour — format, tone, task structure. Fine-tuning is poor at installing facts; it mostly teaches the model to recite a snapshot confidently. Why that follows →
The 2022 Chinchilla result put the compute-optimal ratio at roughly 20 training tokens per parameter — a 70B model on about 1.4 trillion tokens. Models today are deliberately trained far past that point, because over-training a smaller model makes it permanently cheaper to run. The numbers →
Every claim above traces to one of these. They are more readable than their reputation suggests — start with the abstract and the figures.