A designer improves DCRouter's arbiter so that a waiting packet is granted one cycle earlier. Nothing in the specification changes. Overnight, three hundred tests fail. The scoreboard expected packets from input 2 before those from input 3 at output 0, because that was the order the old arbiter happened to produce. The design got better and the testbench called it broken.
What should a check not check, and why does over-checking matter?
The obvious instinct is "the stricter the better": check every output against exactly what the design produced in a known-good run, cycle for cycle. But a check stricter than the specification forbids behaviour the specification allows. It fails legal designs, buries real failures among false ones, and quietly turns the testbench into the specification, so designers learn to match the testbench rather than the document.
Over-specified check | What the specification actually promises | Check instead |
|---|---|---|
Exact output order across inputs | Order within each flow only | Match per (input, output) flow (lesson 2.3) |
Exact latency of 7 cycles | Latency at most 10 cycles | A bound: response within 10 cycles |
A specific value on don't-care bits | Bits are reserved or undefined | Mask them out |
The exact cycle a counter updates | Updated before it is read | Check the value when read |
Internal signal values | Behaviour at the pins | Check at the interface unless the rule is internal |
Results of a golden run | Results allowed by the rules | A model of the rules, not a recording |
// snippet: two ways to check DCRouter's output order
// bad-example idea (in words): "packets from input 2 leave output 0 before packets from input 3"
// That order is the arbiter's choice; the specification does not promise it.
// Right: within one flow (same input and output), order is promised. A scoreboard keeps one
// queue per flow and compares each arriving packet with the oldest expected packet of its flow:
// expected[src][dst].pop_front() must equal the packet that arrived from src at dst.The per-flow scoreboard accepts every legal interleaving at an output and still catches the bugs that matter: a lost, duplicated, corrupted or misordered packet within a flow.
Recording instead of modelling: comparing with the output of a past run captures its accidents as requirements.
Cycle-accurate models where results were specified (lesson 2.3).
Checking internals: assertions on internal signals encode one implementation.
Filling ambiguity with the design's behaviour: the RTL's choice becomes a rule nobody wrote (lesson 2.5).
Too strict | Too loose |
|---|---|
False failures on legal designs | Missed real bugs |
Wasted debug time, eroded trust in failures | False confidence |
Designers optimise for the testbench | The specification is not enforced |
Both are verification bugs. The target is a check that fails exactly when a rule is broken.
Practice | Purpose |
|---|---|
Scoreboards matching per ordering domain (ID, flow) | Accept every legal reordering |
Latency checked as bounds | Allow implementation freedom within the specification |
Checker reviews against the specification | Find rules the testbench invented |
Mutation of legal behaviour | Changing the design legally should not fail tests |
The opening arbiter improvement (illustrative) cost the team three days of triage before someone noticed every failure was an ordering difference across inputs. Replacing the recorded-order comparison with a per-flow scoreboard made all 300 tests pass again; it also, within a week, caught a genuine misordering within one flow that the recorded comparison had been masking among its false alarms.
Any change to arbitration timing that interleaves packets from different inputs in a different order, which the specification allows.
Every test fails with latency 8, although 8 meets the specification; the check should assert a bound of 10.
A check stricter than the specification is a testbench bug: it fails legal designs.
Check exactly what is promised: per-flow order, latency bounds, masked don't-cares, behaviour at the pins.
Every check should cite the rule it enforces.