A model reads input and writes output inside one fixed token budget called the context window. Attention is the mechanism that lets each token use earlier tokens, and its basic cost grows with the square of the length.
Budget item | Example size | Tokens |
|---|---|---|
System prompt | 400 chars of prose | 100 |
Recent chat | 3000 chars of prose | 750 |
Spec extract | 2400 chars of code | 800 |
Answer reserve | 2000 chars of prose | 500 |
Planned total | An EXAMPLE plan, not a real session | 2150 |
Lesson | What it covers |
|---|---|
Input and output sharing one token budget, and how to estimate it. | |
Truncation, output cut-off, and a worked overflow with EXAMPLE cost. | |
Each token looking back, pair counting, and why basic cost is quadratic. | |
The start-end tendency and why huge pastes stay unreliable. | |
A small deterministic program that checks whether a plan fits. |
By 150 tokens. 2150 minus 2000 is 150, so something must be trimmed or summarized before sending.