Four logs land on your desk. dma_check_test hit 9 plan items of 17, dma_zero_len_test hit 4, a directed closure test hit 13, and a test with a modified stream driver hit 10. Your manager asks for one number for the weekly report. Is it 13, the best test? About 9, the average? Or something else?
It is 17 of 17, and none of the four tests reached it alone. This lesson covers how per-test results become one answer, what a merge needs in order to be trusted, and the loop that turns the remaining holes into tests, fixes or reviewed exclusions.
Picture four inspectors walking the same building with the same checklist. Each ticks what they personally saw. The building's inspection is complete when every box is ticked on some sheet, so you combine the sheets by laying them on top of each other, not by averaging how full each one is. A coverage merge does exactly that: for each bin it adds the hit counts from every test, so a bin is covered if any test covered it. The merged number is usually higher than any single test's, and an average of per-test percentages always understates it.
Two more terms. Per-test coverage is what one simulation recorded; it is useful for knowing what a test is for. Merged coverage is the regression's answer, and the only number that signs anything off. Test ranking (or grading) orders tests by how much unique coverage each adds, which tells you which tests a short regression can drop.
The bench's tally is per test, so the union is a simple overlay. Here it is for the four tests in the opening (two from the bench, two scratch tests from the closure lesson):
Test | Items hit | Items it adds to the union so far |
|---|---|---|
dma_check_test | 9 of 17 | len=1, len=2-4, len=5-15, irq_en=1, outcome=done, clear=done, gap=1-3, gap=4+, tlast=on-time |
dma_zero_len_test | 4 of 17 | len=0 |
dma_closure_test (scratch) | 13 of 17 | len=16, len>16, irq_en=0, outcome=err, clear=err, tlast=early |
dma_b2b_test (scratch) | 10 of 17 | gap=0 |
Union | 17 of 17 | No single test above 13 |
The average of the four is 9 of 17. Ranking reads off the last column: dma_zero_len_test and dma_b2b_test each add one item no other test reaches, so dropping either reopens a hole, while dma_check_test's items are all reached by other tests too once the closure tests exist.
On a commercial simulator each test writes a coverage database, and the regression script merges them. The commands below are a sketch; each vendor documents the exact options.
MERGE AND RANK (sketch: check your tool's manual for exact options)
Questa: vcover merge -out merged.ucdb check.ucdb zero_len.ucdb closure.ucdb
vcover report -details merged.ucdb
vcover ranktest check.ucdb zero_len.ucdb closure.ucdb
VCS: urg -dir check.vdb zero_len.vdb closure.vdb -dbname merged
(urg also grades tests by their contribution)
Xcelium: imc merge flow, then a report from the merged run
Tag every database with test name, seed and source revision in the regression manifest.Merge only passing tests. A test that ended with a UVM_ERROR may have exercised a bin while the DUT misbehaved there; its coverage proves nothing. The regression script filters on the verdict before it merges.
The model must match across databases. Covergroups, bins and options must be the same in every test's build, and instance names must be stable (hierarchical names from set_inst_name are). When the model changes, re-run rather than merging old databases under rules you have not checked.
Per-test numbers are for test design. They say what a test is for and whether it still does it. They are not reported upward.
Category | Signature | Action |
|---|---|---|
Volume | Reachable by the random test, just unlucky | More seeds or more jobs per test |
Constraint | A constraint (hard or soft) excludes the value | A test that overrides it, or a deliberate constraint change |
Driver or environment limitation | The bench cannot produce it at all | Fix the driver or add a mode |
Error path | Needs a fault or illegal stimulus | A directed test or fault injection (factory override) |
Model bug | The item cannot be sampled as written | Fix the coverage model, then re-merge |
Unreachable | The spec or configuration makes it impossible | An exclusion with a reason and a reviewer |
The closure lesson puts every one of the DMA's eight holes through this table, and they land in four different rows. That is normal: "run more seeds" closes only the first category.
An exclusion removes an item from the merged denominator by a recorded decision; a waiver is the same idea for a hole the team accepts for now. Both are written down, with a reason a reviewer can check, and both are revisited when the reason changes. One exclusion this bench genuinely needs:
EXCLUSION E-APB-001
Item: top.apb.u_cover.c_nowait (an APB access with no wait state)
Reason: unreachable: dma_dut is built with WAIT_STATES = 1 in every
configuration that ships, so PREADY is always low for one cycle
Reviewed: design lead and verification lead, at the coverage review
Revisit: if any build sets WAIT_STATES = 0The coverage review is a meeting, not a document: the verification engineer walks the merged holes and every exclusion with the designer. Its most valuable outcome is the case where the designer says "that is reachable", which turns an exclusion into a test, and occasionally into a bug.
Code coverage belongs in the same review. Functional coverage complete with code coverage holes means the plan is missing rows (logic nobody wrote a promise for). Code coverage complete with functional holes means stimulus reached every line but not every scenario. Each one finds what the other misses.
+UVM_TESTNAME=dma_check_test
UVM_INFO dma_tb.sv(1531) @ 1305000: uvm_test_top.env.cov [COV] plan holes: len=0 len=16 len>16 irq_en=0 outcome=err clear=err gap=0 tlast=early
+UVM_TESTNAME=dma_zero_len_test
UVM_INFO dma_tb.sv(1530) @ 325000: uvm_test_top.env.cov [COV] plan items hit: 4 of 17
+UVM_TESTNAME=dma_closure_test (scratch run)
UVM_INFO dma_tb.sv(1530) @ 2235000: uvm_test_top.env.cov [COV] plan items hit: 13 of 17
UVM_INFO dma_tb.sv(1531) @ 2235000: uvm_test_top.env.cov [COV] plan holes: len=0 len=1 len=2-4 gap=0
+UVM_TESTNAME=dma_b2b_test (scratch run)
UVM_INFO dma_tb.sv(1530) @ 1195000: uvm_test_top.env.cov [COV] plan items hit: 10 of 17
UVM_INFO dma_tb.sv(1531) @ 1195000: uvm_test_top.env.cov [COV] plan holes: len=0 len=16 len>16 irq_en=0 outcome=err clear=err tlast=earlyEvery one of these runs ended with UVM_ERROR : 0 and UVM_FATAL : 0, so all four may be merged.
The closure test's holes (len=0, len=1, len=2-4, gap=0) are covered elsewhere, which is why it is a closure test: it is designed around the others' holes, not to stand alone.
dma_zero_len_test is the test that closes len=0. Without one line in it, it would fail: the scoreboard's check_phase reports SB_EMPTY (no words compared) as an error, and a zero-length job moves no words. The test demotes that one ID on that one component, on purpose, and its log shows the result as a UVM_INFO.
Now imagine the demotion had been forgotten and the regression script merged every database regardless of verdict. The report would show len=0 covered by a failing test, the failure would sit in a list nobody reads, and the hole would be closed on paper. The opposite shortcut is just as bad: demoting SB_EMPTY globally to make the failure go away would let every future test that silently checks nothing pass. The rule: merge only passing tests, and make every demotion as narrow as the reason for it.
dma_b2b_test hits 10 items, fewer than the closure test's 13. Why does ranking still keep it?Ranking counts unique contribution, not total hits. dma_b2b_test is the only test that reaches gap=0, so dropping it reopens a hole; most of its other items are reached by other tests.
The test's database was not in the merge (wrong list, or the test's build used a different coverage model), or the test never reached the scenario on the pins (for example, its stimulus was blocked by a second constraint, or the coverage model samples something other than what the test exercised). Passing proves the checks held, not that the bin was hit.
c_nowait if the design team adds a WAIT_STATES = 0 build?Its revisit condition has triggered. In that configuration the item becomes reachable, so the exclusion is removed for that build and a test (any register access) covers it.
Next: the union for the DMA reached 17 of 17 with two scratch tests. Closing Coverage: The Holes Random Never Hits shows where each of the eight holes came from and how each was closed.