Every price below is an EXAMPLE parameter, not a real tariff. Teams plug in their own measured numbers and keep the same arithmetic.
Token counts also vary by tokenizer and task mix, so they are measured on team traffic. The calc block uses one EXAMPLE triage attempt.
Line | EXAMPLE value | Cost |
|---|---|---|
Input tokens | 6000 at 3 units per 1M | 6000 x 3 / 10^6 = 0.018 units |
Output tokens | 1500 at 12 units per 1M | 1500 x 12 / 10^6 = 0.018 units |
Total per attempt | Sum of both lines | 0.018 + 0.018 = 0.036 units |
Cost per attempt flatters a weak agent, since failed attempts still bill. Cost per resolved task divides total billed spend over all attempts by the number of resolved tasks.
Human review time usually dominates the token bill. Log review minutes per task beside the token cost so the comparison stays honest.
EXAMPLE prices: 3 units per 1M input tokens, 12 units per 1M output tokens
per-attempt tokens (stated EXAMPLE): 6000 in + 1500 out
cost per attempt = 6000 x 3 / 10^6 + 1500 x 12 / 10^6 = 0.018 + 0.018 = 0.036 units
valid runs = 200 - 30 invalid = 170
success rate = 80 / 170 = 8 / 17 = about 0.47
assuming all 200 attempts incurred the stated cost, total spend = 200 x 0.036 = 7.2 units
cost per resolved task = 7.2 / 80 = 0.09 units
conditional valid-case success is 80/170, but it must not erase billed invalid attemptsOutcome over 200 attempts | Count | Share |
|---|---|---|
Resolved | 80 | 40% |
Attempted, check did not pass | 90 | 45% |
Invalid setup, excluded | 30 | 15% |
Report both 80/200 overall success and 80/170 valid-case success, with exclusion reasons. The table excludes invalid setups from the latter accuracy denominator only. All billed work, including invalid attempts, stays in total spend. If an invalid setup failed before billing, record its actual zero spend instead.
The program below recomputes the cost numbers from the same EXAMPLE parameters. It uses plain arithmetic, with no model involved.
import sys
input_tokens = 6000
output_tokens = 1500
input_price = 3.0
output_price = 12.0
attempts = 200
invalid = 30
resolved = 80
cost_attempt = (input_tokens * input_price + output_tokens * output_price) / 1000000.0
valid = attempts - invalid
rate = resolved / valid
total_spend = cost_attempt * attempts
cost_resolved = total_spend / resolved
if abs(cost_attempt - 0.036) > 1e-9:
print("FAIL: attempt cost arithmetic off")
sys.exit(1)
if valid != 170:
print("FAIL: valid base arithmetic off")
sys.exit(1)
if abs(rate - 80.0 / 170.0) > 1e-12:
print("FAIL: rate arithmetic off")
sys.exit(1)
if abs(cost_resolved - 0.09) > 1e-9:
print("FAIL: resolved cost arithmetic off")
sys.exit(1)
print("PASS: 7.2 units across all attempts; 0.09 units per resolved task; valid-case rate about 0.47")Report mean and a high percentile together, since agents have long tails. A mean of 45 s with a p95 of 120 s plans capacity differently than 45 s alone.
Parallel workers cut wall time but not total cost. 200 tasks at 45 s each need 9000 s of work; 8 workers finish in 1125 s, about 19 min.
0.09 units: 200 x 0.036 = 7.2 units in total, then 7.2 / 80. Dividing by the valid-case rate (0.036 / 0.47, about 0.077) would hide the 30 billed invalid attempts.
1125 s, about 19 min. Total work is 9000 s divided by 8 workers.