Question 1: how many tokens is this input

Tests whether the candidate estimates before sending. Strong answers use a rule of thumb and name its limits.

A common approach to tokenization is subword units. For English prose a rough rule is about 3 to 4 characters per token with common tokenizers, stated as approximate and tokenizer-dependent.

1
2
3
4

example document: 12000 characters of English prose
rule of thumb (approximate, tokenizer-dependent): about 4 characters per token
estimate = 12000 / 4 = 3000 tokens (planning range about 3000 to 4000)
same characters as hex at about 2 characters per token: 12000 / 2 = 6000 tokens

Code, numbers, hex and non-Latin text carry fewer characters per token. The only exact count comes from the tokenizer that will process the input.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def estimate_tokens(chars, chars_per_token):
    return chars / chars_per_token


def fits(tokens, budget):
    return tokens <= budget


bad = []
if estimate_tokens(12000, 4) != 3000.0:
    bad.append("prose estimate should be 3000")
if estimate_tokens(12000, 3) != 4000.0:
    bad.append("dense estimate should be 4000")
if not fits(3000.0, 8000):
    bad.append("3000 should fit budget 8000")
if fits(9000.0, 8000):
    bad.append("9000 should exceed budget 8000")
if bad:
    print("FAIL: " + "; ".join(bad))
    raise SystemExit(1)
print("PASS: 12000 chars estimate 3000 to 4000 tokens, budget check holds")

The program above is a plain estimator, not a real tokenizer. It proves the planning arithmetic used in the model answers.

Question 2: what breaks as context grows

Tests understanding of attention cost and long-context reliability. Strong answers separate cost from quality.

In the basic form, attention cost grows with the square of the sequence length. Efficient variants exist, so this is a statement about the basic form only.

1
2
3
4

basic attention cost grows with the square of the sequence length (efficient variants exist)
length 4000: proportional to 4000^2 = 16000000
length 8000: proportional to 8000^2 = 64000000 = 4 x 16000000
doubling the length quadruples the pairwise cost

Long-context use is less reliable than short-context use. Models tend to use information at the start and end of long inputs better than the middle, a tendency reported in studies, not a law.

Question 3: what stays in context

Tests judgment about truncation and summarization. Strong answers keep the task, the constraints and the freshest evidence.

Summarize stale history into decisions and keep raw text for disputed facts. Anything that differs between models or tools must be said to vary, so measure the short version against the long one.

Common mistakes

Related topics

• Cost estimation questions

• Cost arithmetic

• Tokens and tokenization

• Context window and attention for users