Building this course's DCRouter testbenches produced a steady stream of failures that were not DCRouter's fault. A credit audit failed the clean design until the sinks were drained first (lesson 4.8). An arbitration checker failed the clean design in 17 of 30 seeds (lesson 5.5). A formal property failed at step 2 because it ignored the reset cycle (lesson 7.9). Sending all three to the design team's queue would have cost the designers days and the verifiers their credibility.
How do you know the bug is in the design?
The obvious assumption is that a failing check found a design bug: that is what checks are for. Testbenches are code too, often as large as the design, written faster and reviewed less. A failure is a disagreement between the design and the testbench's model of the specification; either side can be wrong, and so can the specification.
Question | Points to the testbench if | Points to the design if |
|---|---|---|
Does the clean (or previous good) design pass the same test? | It fails too | Only the new design fails |
Does the driver obey the interface protocol? | A protocol assertion on the driver's side fires | The design's outputs break the protocol |
What does the specification say, exactly? | The checker expects more than the specification promises (lesson 2.7) | The design does something the specification forbids |
Is the failure simulator- or ordering-dependent with the same seed? | Only some simulators or orderings fail: a testbench race | Consistent across simulators |
Does another, independent check agree? | No other check sees anything | A second checker or a formal property also fails |
Does the checker's model make assumptions the pins cannot confirm? | Yes, as with hidden timing (lesson 5.5) | No |
Where | Symptom | Cause | How it was told apart |
|---|---|---|---|
Credit audit (4.8) | Clean design failed the audit | Sinks still held flits and credits when the audit began | Failed on the clean design |
Arbitration checker v1 (5.5) | Priority violations on the clean design | Grant time assumed from first-flit time | Failed on the clean design across seeds |
Arbitration checker v2 (5.5) | Priority violations with credit bugs | Its own credit count was wrong when the design's was | The credit audit, an independent check, failed first |
Grey-box FIFO checker (3.4) | Could never fail on BUG_1 | Checked the design's own full flag | Planted bug not caught |
Rotation property (7.9) | Formal failure at step 2 on the clean design | Ignored the reset cycle | Failed on the clean design |
Constraints demo (4.3) | Lengths not shaped as intended | := instead of :/ in a dist | Histogram did not match the intent |
Three habits caught every one of these before it reached anyone else: running every new check on the clean design (across seeds), running it against planted bugs (lesson 5.8), and comparing it with an independent check or the intended distribution.
Show the rule: the specification sentence the design breaks.
Show a minimal reproduction (lesson 13.4) and the waveform at the first wrong cycle.
Show the clean comparison: the previous good design or a reference model passing.
Say what you ruled out: driver protocol, checker assumptions, simulator.
Practice | Purpose |
|---|---|
Protocol assertions on testbench drivers as well as the design | Testbench protocol bugs caught at once |
New checkers validated on the clean design and planted bugs | False failures removed before release |
Testbench bugs tracked in the same database | Their cost visible; their causes learned from |
Bug reports with rule, reproduction and evidence | Designers' time spent on real design bugs |
A team (illustrative) sent every failure straight to the designers. After weeks in which most turned out to be testbench bugs, designers began to treat verification reports as noise, and a real design bug report sat unread for nine days. Credibility is a verification asset; it is spent by every unchecked report.
Mostly running the new check on the clean design, across seeds; for the grey-box FIFO checker, running it against the planted bug it should catch; for arbitration checker v2, an independent check (the credit audit); for the constraints demo, comparing the histogram with the intended distribution.
A testbench race or uninitialised state (non-determinism), more likely than a design bug.
Many failures are testbench bugs; a failure is a disagreement, not a verdict.
Separate them with the clean-design comparison, protocol checks, the specification and independent checks.
Validate checkers on clean designs and planted bugs, and report design bugs with evidence.