A new interconnect (illustrative) passes its full regression: every transaction arrives, every result is correct, every assertion is silent. On silicon, one processor's memory latency is ten times its neighbours' under load. The arbiter favours the lowest-numbered requester whenever several compete. No functional check could fail, because nothing was ever wrong, only slow.
Why do performance bugs escape functional verification, and how are they found?
The obvious view is that correct means correct results, and that a scoreboard matching every output proves the design works. Performance bugs deliver the right results at the wrong time: late, unfairly ordered, or at a fraction of the specified rate. A scoreboard compares what came out, not when, so it passes. Worse, many performance bugs only appear under heavy load, and light test traffic hides them completely.
Performance property | Example bug | Check |
|---|---|---|
Throughput | A credit lost per collision (DCRouter BUG_9) slowly reduces the rate | Measure sustained rate against the specification |
Latency | A queue served last under load | Bound the latency of each request; track the maximum |
Fairness | A pointer that never moves (DCRouter BUG_5) | Bound how long a waiting requester can be passed over |
Starvation freedom | A low-priority class never served | Every request eventually granted, within a bound |
Four requesters share one output. Each holds one request until it is granted. A round-robin arbiter is compared with one whose pointer never moves, the planted DCRouter bug BUG_5, at light and heavy load.
"""A performance bug: an arbiter that is functionally correct but unfair.
Four requesters share one output that grants one request per cycle. As in hardware,
each requester holds a single request until it is granted, then raises its next one
with probability `load` per cycle. Two arbiters:
round-robin: the priority pointer moves past each winner (DCRouter rule R6)
stuck pointer: the pointer never moves, so requester 0 always wins first (Bug Zoo BUG_5)
Both grant only real requests and never lose one: every functional check passes.
The model reports, per requester, the share of grants and the longest wait in cycles
(a request still pending at the end counts up to the end), at a light load and a heavy one.
"""
import random
def run(stuck, load, cycles=200000, seed=5):
rng = random.Random(seed)
waiting = [None] * 4 # cycle the pending request was raised, or None
grants = [0] * 4
worst = [0] * 4
ptr = 0
for t in range(cycles):
for i in range(4):
if waiting[i] is None and rng.random() < load:
waiting[i] = t
for k in range(4):
i = (ptr + k) % 4
if waiting[i] is not None:
worst[i] = max(worst[i], t - waiting[i])
waiting[i] = None
grants[i] += 1
if not stuck:
ptr = (i + 1) % 4
break
for i in range(4): # a request still pending at the end has waited too
if waiting[i] is not None:
worst[i] = max(worst[i], cycles - waiting[i])
total = sum(grants)
return [g / total for g in grants], worst
results = {}
for load in (0.1, 0.9):
print(f"load {load}: chance per idle cycle that a requester raises a new request")
for name, stuck in (("round-robin ", False), ("stuck pointer", True)):
share, worst = run(stuck, load)
results[(stuck, load)] = (share, worst)
print(f" {name} share " + " ".join(f"{s:5.2f}" for s in share) + " longest wait " + " ".join(f"{w:6d}" for w in worst))
lo = results[(True, 0.1)][1]
rr = results[(False, 0.9)][1]
st_share, st_worst = results[(True, 0.9)]
assert max(rr) <= 3 and max(lo) < 10 and st_worst[3] > 100000 and st_share[3] < 0.01
print("PASS: at light load the stuck pointer looks fair; under heavy load it starves requesters 2 and 3 while round-robin keeps every wait within 3 cycles")It prints:
load 0.1: chance per idle cycle that a requester raises a new request
round-robin share 0.25 0.25 0.25 0.25 longest wait 3 3 3 3
stuck pointer share 0.26 0.25 0.25 0.24 longest wait 0 5 8 9
load 0.9: chance per idle cycle that a requester raises a new request
round-robin share 0.25 0.25 0.25 0.25 longest wait 3 3 3 3
stuck pointer share 0.90 0.10 0.00 0.00 longest wait 0 98 6026 199998
PASS: at light load the stuck pointer looks fair; under heavy load it starves requesters 2 and 3 while round-robin keeps every wait within 3 cyclesHow to read it: at light load the two arbiters are indistinguishable, with even shares and waits under ten cycles; light functional tests cannot tell them apart. At heavy load, round-robin still gives everyone 25% and never makes anyone wait more than three cycles. The stuck pointer gives requester 0 90% of the grants; requester 3 raised its first request and was still waiting at the end of the 200,000-cycle run. Every grant was legal, so every functional check passes.
Specify the numbers: throughput, latency bounds and fairness limits must be in the specification, or nobody can check them (Part 2).
Load the design: heavy and adversarial traffic, not just light functional tests.
Monitor and assert: track per-requester latency and grants; assert bounded waiting (Part 5).
Compare with a model: a performance model predicts the expected rate; a large gap is a bug.
Practice | Purpose |
|---|---|
Performance tests with saturating traffic | Expose behaviour only visible under contention |
Latency and fairness monitors in the testbench | Turn timing into checked numbers |
Bounded-wait assertions on arbiters | Fail the first time a requester waits too long |
Performance models before RTL | Give the numbers the RTL must meet |
DCRouter's two performance bugs, a stuck pointer (BUG_5) and three credits instead of four after reset (BUG_10), both pass the smoke test of lesson 0.7 and any test that only compares data. The course's reference regression catches them only with checks written for performance: a fairness check that no input wins twice in a row while others of the same priority wait, and a credit count with the sink stalled.
With no contention there is nothing to arbitrate; each request is granted at once, so the unfairness never shows.
Only performance checks: a throughput measurement against the expected rate, or a check of the credit count; data checks all pass.
Performance bugs deliver correct results too slowly or unfairly, so functional checks pass.
Many appear only under heavy load: light tests cannot tell a fair arbiter from a starving one.
Specify throughput, latency and fairness, load the design, and check the numbers.