A language model maps input text to a distribution over next tokens and samples one token at a time. This topic follows that mechanism from training through sampling.
Term | Meaning |
|---|---|
Next-token prediction | Scoring each vocabulary item as a possible continuation of the text so far. |
Training | Setting the model numbers from a large text collection before use. |
Inference | Running the fixed model on a new prompt to score and sample tokens. |
Parameters | The stored numbers set by training and read at inference. |
Temperature | A sampling control that sharpens or flattens the next-token distribution. |
Knowledge cutoff | The point where the training collection ends, after which events are unknown. |
Lesson | What it covers |
|---|---|
The scoring step and the loop that builds text token by token. | |
What training sets, what inference reuses, and what differs in practice. | |
Stored numbers at an intuitive level, with EXAMPLE storage arithmetic. | |
Why fluent text can be wrong and how cutoffs limit freshness. | |
Temperature scaling with a runnable fixed-table sampling model. |
Sampling explains it. Scores may be similar, but each run draws a different token at some step and the continuations diverge.