What an LLM actually does: next-token prediction
A large language model is, mechanically, a next-token predictor. Given a sequence of tokens, it outputs a probability distribution over the next token. That's it. Everything else — reasoning, coding, translation — is emergent behavior from doing this extremely well at scale over trillions of tokens.
Tokens, not words
Models don't see characters or words; they see tokens — sub-word chunks from a fixed vocabulary (~100k–256k for modern models). "tokenization" might be 2 tokens; a rare word or code symbol might be several. Rule of thumb: ~4 characters ≈ 1 token, and 100 tokens ≈ 75 words in English (so ~1.3 tokens per word — English text is a bit more than one token per word).
The Transformer, in one breath
The architecture is the Transformer (Vaswani et al., 2017). Its key mechanism is self-attention: every token can look at every other token in the context and weigh how relevant each is. Stack many attention + feed-forward layers, train to predict the next token, and you get an LLM. You don't need to derive attention math for this role — but you must know what it does and why it's expensive (attention cost grows with the square of sequence length).
◆ Lock it in
- LLMs predict the next token from a probability distribution; capability is emergent from scale.
- Models operate on tokens (~4 chars each), not words or letters — this drives cost, limits, and quirks.
- The engine is the Transformer; self-attention lets every token weigh every other; cost scales ~quadratically with length.