Parameters are the numbers the model reads at every prediction step. Training sets them, and inference leaves them unchanged.
Each number has a small fixed role in transforming token signals into scores. Prediction quality comes from all of them acting together, not from any single value.
Where numbers live | When they change | What they do |
|---|---|---|
Stored with the model | During training only | Shape every next-token score |
Loaded for inference | Never during a session | Fixed mapping from text to scores |
Counts differ between models, so every number below is a labelled EXAMPLE parameter. The arithmetic pattern is what lasts: storage equals count times bytes per number.
Precision also varies by model and deployment, so treat 2 bytes per number as a stated assumption. Measure the real deployment before sizing hardware.
EXAMPLE parameters, stated illustration only, checkable by hand
count = 1000000000 parameters (assumed EXAMPLE value)
bytes per number = 2 B (assumed EXAMPLE precision)
storage = 1000000000 x 2 B = 2000000000 B
result: 2000000000 B, about 2 GB in decimal unitsMore numbers can represent finer patterns, but nothing in the count guarantees truth. A larger store of patterns still predicts likely text rather than verified fact.
Small and large models fail in the same observable way when evidence is missing. Both produce fluent continuations that checks must still verify.
More parameters change | More parameters do not change |
|---|---|
Finer distinctions the scores can express | Whether any single claim is true |
Larger storage and compute needs | The need to verify output before use |
A frequent mistake is reading size as a correctness guarantee. Size changes capacity, while verification decides acceptance.
2000000000 bytes, about 2 GB decimal. One billion times 2 bytes equals two billion bytes.
Neither. Size changes capacity, and every claim still needs verification against tests or sources.