Tests whether the candidate turns tokens into money with a stated formula. Strong answers label every price as an example assumption.
Never state prices of any real model or tool as fact. Say assume, show the formula, substitute, and give the result with units.
EXAMPLE parameters (not real prices): input 3 units per million tokens, output 12 units per million
job: 40000 input tokens, 2000 output tokens
input cost = 40000 / 1000000 x 3 = 0.04 x 3 = 0.12 units
output cost = 2000 / 1000000 x 12 = 0.002 x 12 = 0.024 units
total = 0.12 + 0.024 = 0.144 units
input share = 0.12 / 0.144 = 5/6, output share = 0.024 / 0.144 = 1/6, sum 5/6 + 1/6 = 1INPUT_PRICE = 3.0 # EXAMPLE units per million input tokens, not a real price
OUTPUT_PRICE = 12.0 # EXAMPLE units per million output tokens, not a real price
def job_cost(in_tokens, out_tokens):
return in_tokens / 1000000 * INPUT_PRICE + out_tokens / 1000000 * OUTPUT_PRICE
def single_retry_cost(base, retry_prob):
# at most ONE full-cost retry: expected cost = base x (1 + p)
return base * (1 + retry_prob)
bad = []
base = job_cost(40000, 2000)
if abs(base - 0.144) > 1e-9:
bad.append("base should be 0.144")
if abs(single_retry_cost(base, 0.2) - 0.1728) > 1e-9:
bad.append("single-retry total should be 0.1728")
if bad:
print("FAIL: " + "; ".join(bad))
raise SystemExit(1)
print("PASS: base 0.144 units, one-retry expected 0.1728 units")The program recomputes the worked example above. It proves the cost formula and the single-retry expectation, not any real price.
Tests reading the shares before reaching for levers. Here input tokens dominate at five sixths of the total.
For at most one full-cost retry, expected cost is the base times one plus the retry chance: 1.2 at chance 0.2, giving 0.1728 units. If retries can repeat, expected attempts are 1 / (1 - p) = 1.25 at p = 0.2.
Lever | Attacks | Watch out for |
|---|---|---|
Shorter inputs | The five-sixths input share | Dropping the evidence the task needs. |
Shorter outputs | The per-token output weight | Truncating the reasoning the answer needs. |
Fewer retries | The single-retry overhead of one fifth | Accepting the first draft unchecked. |
Tests judgment across lessons instead of repeating them. Strong answers point at the reducing-tokens and throughput lessons for the technique details.
Measure the split first, then cut the largest share. Report the new total with the same example prices so the two answers compare directly.
0.144 units: 0.12 input plus 0.024 output. Input is five sixths of the total.
1.2 times the base: one plus the retry chance, so 0.1728 units. If retries can repeat instead, the factor is 1 / (1 - 0.2) = 1.25.