After Part 5, DCRouter's testbenches catch 22 of the 24 Bug Zoo bugs. The other two, BUG_16 (PKT_CNT3 not cleared) and BUG_23 (a disabled port still accepts traffic), escaped every test, and nothing in any log hinted at them. Every test passed. A pass/fail result cannot say what a test never did; something has to record what happened, so that what did not happen becomes visible.
What does coverage measure, and what does it prove?
The obvious reading of a coverage number is "how much of the design is verified": 100% coverage, done. Coverage measures something narrower: whether certain events happened during simulation. It says nothing about whether a check looked at the result (lesson 3.3), or whether the events measured are the ones that matter. A coverage report is a list of what the stimulus did, and, more usefully, of what it did not.
Kind | Measures | Written by |
|---|---|---|
Code coverage | Which lines, branches, conditions, toggles and FSM states of the RTL were executed | The simulator, automatically (lesson 6.2) |
Functional coverage | Which situations from the verification plan occurred: values, combinations, sequences | Verification engineers, from the plan (lesson 6.3) |
Assertion coverage | Which assertions were triggered (non-vacuously) and which cover properties occurred | Alongside the assertions (lesson 5.8) |
All three record activity. None of them records that a check compared the result. Coverage and checking answer two different questions: checks ask "was what happened right?", coverage asks "what happened?"
100% coverage of this | Proves | Does not prove |
|---|---|---|
Code | Every line and branch executed at least once | That any check saw the result; that the right values or combinations reached it |
A functional model | Every situation in the model occurred | That the model lists the situations that matter; that checks were active |
Assertions | Every assertion was triggered | That the assertions express the whole specification |
Escape | What no test did | A coverage bin that stays empty |
|---|---|---|
BUG_23 | Disable a port | "CTRL port k disabled" |
BUG_16 | Clear PKT_CNT3 | "PKT_CNT3 cleared" |
BUG_23, again | Trigger the R1 assertion (lesson 5.3) | A cover on R1's condition, "a port is disabled": empty, so the assertion was never tested |
Both bugs live behind actions the stimulus never took. A coverage model listing those actions shows two empty bins after every run. Lesson 6.9 builds that model and uses its holes to write the test that catches both.
Read holes, not percentages: the list of what never happened is the useful output.
Pair every covered situation with a check: coverage without checking is activity, not evidence.
Write functional coverage from the plan, not from the RTL: it should describe what matters, not what exists.
Merge across seeds and tests: closure is a property of the whole regression (lesson 4.9).
Practice | Purpose |
|---|---|
Coverage goals in the verification plan | Agreed measure of what must be exercised (Part 9) |
Merged regression coverage reviewed weekly | Holes drive new tests and constraint changes |
Coverage as one sign-off criterion among several | With pass rates, bug trends and reviews (Part 14) |
A team (illustrative) reached 100% functional coverage and signed off. Later a bug was found in a covered situation: the coverage point sampled the input request, but the response check for that request type had been disabled during debug and never re-enabled. Coverage proved the situation happened; nothing proved it was checked. Pairing coverage with assertion coverage on the checks themselves, or a mutation run, would have shown it.
No test ever cleared PKT_CNT3 or disabled a port, so the bugs were never activated; a test can only fail on something it did.
Fully covered; coverage measures that it happened, not that anything checked it.
Coverage measures what was exercised, never what was verified.
Its most useful output is the list of holes: what the tests never did.
Pair coverage with checking, and write functional coverage from the plan.