Lesson 0.2 showed that a bug costs more the later it is found: minutes in a designer's test, weeks in the lab, a respin at a customer. The natural conclusion is "verify as much as possible". A manager with a budget asks the sharper question: how much verification is worth paying for? Every engineer-week costs money too, and at some point another week finds less than it costs.

Why find bugs early, and how much verification is enough, in money?

The obvious answer, and why it breaks

One obvious answer is "never stop": escapes are so expensive that more verification is always justified. The other is "verify until the schedule ends". The first ignores that each extra week finds fewer bugs (lesson 12.3); the second ignores the bugs left behind. Expected cost has two parts, the verification spent and the escapes not prevented, and their sum has a minimum.

The idea: verification cost plus escape cost

• Verification cost rises steadily with effort.

• Escape cost falls with effort, because each week finds some of the remaining bugs; each escaped bug costs the weighted average of where it would be found later.

• Total cost is high with too little verification, lowest at an optimum, and rises again slowly with too much.

• The optimum moves with the cost of an escape: safety-critical or high-volume parts push it far to the right.

Build it: one block, 60 bugs

An illustrative block with 60 bugs; effort finds each bug with a probability that rises with engineer-weeks; escaped bugs are found later at emulation, lab or customer costs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

"""Cost of bugs by stage: how much verification is worth paying for?

A block holds 60 bugs (illustrative). Verification effort E (engineer-weeks) finds each bug
with probability 1 - exp(-E / 40). Every bug not found pre-silicon is found later, at a
stage-dependent cost: 70% in emulation or system validation ($20k each), 25% in the lab on
silicon ($200k each, including workarounds and debug), 5% at customers ($2M each, including a
respin share and reputation). Verification costs $5k per engineer-week. The model prints
total expected cost against effort and the effort that minimises it.
"""
import math

BUGS, SCALE, WEEK = 60, 40.0, 5_000
escape_cost = 0.70 * 20_000 + 0.25 * 200_000 + 0.05 * 2_000_000

def costs(e):
    escaped = BUGS * math.exp(-e / SCALE)
    return e * WEEK, escaped * escape_cost, escaped

efforts = [0, 25, 50, 100, 150, 200, 300, 400]
print("effort (eng-weeks)  verification $   escapes   escape cost $     total $")
rows = []
for e in efforts:
    v, x, n = costs(e)
    rows.append((e, v, x, v + x))
    print(f"{e:18d}  {v:14,.0f}  {n:8.1f}  {x:14,.0f}  {v + x:12,.0f}")
best = min(range(0, 601), key=lambda e: sum(costs(e)[:2]))
v, x, n = costs(best)
print(f"average cost of an escaped bug: ${escape_cost:,.0f}")
print(f"cheapest total at {best} engineer-weeks: ${v + x:,.0f}, with {n:.1f} bugs escaping")
assert 100 < best < 300 and rows[0][3] > 3 * (v + x)
print(f"PASS: under-verifying costs more than three times the optimum; past {best} engineer-weeks each extra week costs more than the escapes it prevents")

It prints:

effort (eng-weeks)  verification $   escapes   escape cost $     total $
                 0               0      60.0       9,840,000     9,840,000
                25         125,000      32.1       5,266,972     5,391,972
                50         250,000      17.2       2,819,207     3,069,207
               100         500,000       4.9         807,716     1,307,716
               150         750,000       1.4         231,415       981,415
               200       1,000,000       0.4          66,301     1,066,301
               300       1,500,000       0.0           5,442     1,505,442
               400       2,000,000       0.0             447     2,000,447
average cost of an escaped bug: $164,000
cheapest total at 156 engineer-weeks: $979,180, with 1.2 bugs escaping
PASS: under-verifying costs more than three times the optimum; past 156 engineer-weeks each extra week costs more than the escapes it prevents

How to read it: with no verification all 60 bugs escape and cost $9.8M. Fifty engineer-weeks still leave 17 escapes and $3.1M. The minimum, about $0.98M, is at 156 engineer-weeks, with about one bug expected to escape. Beyond it, verification keeps finding bugs, but each extra week costs more than the escapes it prevents. Every number here is illustrative; the shape is not.

What moves the optimum

How real teams use cost thinking

What breaks

A team (illustrative) cut block-level verification by half to meet a schedule, reasoning that chip-level tests would catch what was missed. Chip-level debug of block bugs took three times longer per bug, and two bugs reached silicon, each needing a software workaround that cost performance. The model's left side is this decision: saving verification weeks that the escapes cost back many times over.

Key takeaways

• Total cost is verification spent plus escapes not prevented; it has a minimum.

• Too little verification costs many times the optimum; too much costs more than it prevents.

• The optimum moves with escape cost, technique quality and the number of bugs.

Related topics

• Part 12 overview: Bug Economics