dma_check_test passes cleanly: the scoreboard and the comparator both agree on 14 words out of 14. Its coverage line says plan items hit: 9 of 17, followed by eight holes. Someone in the stand-up says what everyone says: "Run more seeds overnight."

That would close exactly one of the eight, and only with luck. The others are out of reach of random stimulus for four different reasons, one of them a limitation in the stream driver that no checker could ever have reported. This lesson takes the holes one at a time, from the real log to the closing test.

The idea: random stimulus explores only the space you gave it

Random stimulus is a fishing net. More casts (seeds) catch more of the fish inside the area the net covers, but no number of casts catches a fish outside it, or smaller than the mesh. Constraints define the area. The driver defines the mesh: some shapes of traffic it simply cannot produce. Error paths are fish in a different lake: the random test was written to make the DUT succeed. Closure is the work of knowing which situation each hole is in, because each needs a different fix.

The picture: eight holes, four causes

Build it: the holes, one at a time

Where the constraints come from

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// snippet: from the course DMA bench
// One DMA job as a randomizable object. A test can randomize it, or put one in
// config_db so a test further down the hierarchy picks it up.
class dma_job_cfg extends uvm_object;
  rand bit [31:0]   key;
  rand int unsigned len;
  rand bit          irq_en;
  constraint len_c { len inside {[1:16]}; }
  constraint irq_c { soft irq_en == 1; }

  `uvm_object_utils(dma_job_cfg)
  function new(string name = "dma_job_cfg");
    super.new(name);
  endfunction

  function string convert2string();
    return $sformatf("key=%08h len=%0d irq_en=%0b", key, len, irq_en);
  endfunction
endclass

len_c allows 1 to 16, so len=0 and len>16 are excluded by a hard constraint. irq_c is soft, so the solver honours it whenever nothing conflicts, which in a plain randomize() is always: irq_en=0 is excluded just as firmly, only more politely.

len=16: a volume hole

16 is inside len_c, and LEN is uniform over 16 values, so each job has a 1-in-16 chance. With three jobs per test the chance of seeing it at least once is 1 − (15/16)³, about 18%. The bench runs with the simulator's default seed, so every run of dma_check_test draws the same three jobs (lengths 10, 1 and 3, with the keys shown in the log). More seeds, or more jobs per test, close this one; nothing else in this list.

len=0: closed by a directed test

The spec says a zero-length job is legal and finishes at once with DONE. A constraint change is the wrong fix: allowing 0 in len_c would make every random job a candidate for doing nothing. A directed test says exactly what the plan row says:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

// snippet: from the course DMA bench
// Closing a coverage hole: random jobs never use LEN = 0 (dma_job_cfg allows
// 1 to 16), so the len=0 plan item stays empty. This directed test closes it:
// a zero-length job must finish at once with DONE and no data.
class dma_zero_len_test extends dma_base_test;
  `uvm_component_utils(dma_zero_len_test)
  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction
  function void build_phase(uvm_phase phase);
    dma_env::type_id::set_type_override(dma_env_check::get_type());
    super.build_phase(phase);
  endfunction
  // No data moves, so the scoreboard's "nothing was compared" error is expected
  // here, and only here: demote that one ID on that one component, on purpose.
  function void end_of_elaboration_phase(uvm_phase phase);
    super.end_of_elaboration_phase(phase);
    env.sb.set_report_severity_id_override(UVM_ERROR, "SB_EMPTY", UVM_INFO);
  endfunction
  task run_phase(uvm_phase phase);
    bit [31:0] status, count;
    phase.raise_objection(this);
    reg_write(KEY, 32'h1357_9BDF);
    reg_write(LEN, 0);
    reg_write(CTRL, 32'h3);                  // GO with no data to move
    wait (irq_vif.irq === 1'b1);
    reg_read(STATUS, status);
    reg_read(COUNT, count);
    if (status[2:0] !== 3'b010) `uvm_error("ZERO", $sformatf("expected DONE only, STATUS=%08h", status))
    if (count !== 0)            `uvm_error("ZERO", $sformatf("expected COUNT=0, got %0d", count))
    reg_write(STATUS, 32'h2);
    `uvm_info("ZERO", "zero-length job finished at once with DONE", UVM_LOW)
    #50ns;
    phase.drop_objection(this);
  endtask
endclass

+UVM_TESTNAME=dma_zero_len_test
UVM_INFO dma_tb.sv(1718) @ 275000: uvm_test_top [ZERO] zero-length job finished at once with DONE
UVM_INFO dma_tb.sv(407) @ 325000: uvm_test_top.env.sb [SB_EMPTY] no words were compared: the test checked nothing
UVM_INFO dma_tb.sv(1530) @ 325000: uvm_test_top.env.cov [COV] plan items hit: 4 of 17
UVM_INFO dma_tb.sv(411) @ 325000: uvm_test_top.env.sb [SB] 0 words matched, 0 mismatched
UVM_ERROR :    0
UVM_FATAL :    0

• The test checks the DUT itself: STATUS must be DONE alone and COUNT must be 0. With no data moving, the scoreboard has nothing to check, so the test carries the check.

• SB_EMPTY prints as UVM_INFO. set_report_severity_id_override on env.sb demotes that one ID, on that one component, in this one test. Every other test still fails if it compares nothing.

• 4 of 17: len=0, irq_en=1, outcome=done and clear=done. The test is narrow on purpose.

len>16 and irq_en=0: constraint holes, and the surprises behind them

Overriding the constraints looked like a two-line test. The first scratch attempt sent a 20-word job through run_job and died at once:

+UVM_TESTNAME=dma_closure_test   (scratch run, first attempt)
UVM_FATAL dma_tb.sv(670) @ 1005000: uvm_test_top [RAND] axis_pkt_seq randomize failed

len>16 was blocked by a second constraint in a different class: axis_pkt_seq also limits its packet size n to 1 to 16, and run_job randomizes it with n == n_words. irq_en=0 had a surprise of its own: run_job waits for the interrupt, which never comes with IRQ_EN off, so that job would hang until the timeout. The closing test therefore polls STATUS for the IRQ-off job, and turns off the packet constraint for the long one. It also closes the error-path holes with a short packet: LEN = 8 but only 3 words, the last carrying TLAST. This is a scratch test, not part of the course bench:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

// snippet: scratch closure test (not in the course bench)
class dma_closure_test extends dma_base_test;
  `uvm_component_utils(dma_closure_test)
  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction
  function void build_phase(uvm_phase phase);
    dma_env::type_id::set_type_override(dma_env_check::get_type());
    super.build_phase(phase);
  endfunction
  task run_phase(uvm_phase phase);
    dma_job_cfg  job;
    axis_pkt_seq pkt;
    bit [31:0]   status;
    phase.raise_objection(this);
    // irq_en=0 and len=16: no interrupt will come, so poll STATUS
    reg_write(KEY, 32'h0BAD_F00D);
    reg_write(LEN, 16);
    reg_write(CTRL, 32'h1);                         // GO, IRQ_EN = 0
    pkt = axis_pkt_seq::type_id::create("pkt");
    if (!pkt.randomize() with { n == 16; }) `uvm_fatal("RAND", "randomize failed")
    pkt.start(env.in_agt.sqr);
    do reg_read(STATUS, status); while (status[2:1] == 2'b00);
    if (status[2:1] !== 2'b01) `uvm_error("CLOSE", $sformatf("len=16 polled job: STATUS=%08h", status))
    reg_write(STATUS, 32'h2);
    // len>16: axis_pkt_seq also limits n to 1..16, so switch that constraint off
    reg_write(LEN, 20);
    reg_write(CTRL, 32'h3);
    pkt = axis_pkt_seq::type_id::create("pkt20");
    pkt.n_c.constraint_mode(0);
    if (!pkt.randomize() with { n == 20; }) `uvm_fatal("RAND", "randomize failed")
    pkt.start(env.in_agt.sqr);
    wait (irq_vif.irq === 1'b1);
    reg_read(STATUS, status);
    reg_write(STATUS, 32'h6);
    if (status[2:1] !== 2'b01) `uvm_error("CLOSE", $sformatf("len=20 job: STATUS=%08h", status))
    // outcome=err, clear=err, tlast=early: LEN = 8 but a 3-word packet
    job = dma_job_cfg::type_id::create("job");
    job.key = 32'h1234_5678; job.irq_en = 1; job.len = 8;
    run_job(job, 3, status);
    if (status[2:1] !== 2'b10) `uvm_error("CLOSE", $sformatf("short packet: STATUS=%08h", status))
    #100ns;
    phase.drop_objection(this);
  endtask
endclass

+UVM_TESTNAME=dma_closure_test   (scratch run)
UVM_INFO closure_tb.sv(33) @ 885000: uvm_test_top [CLOSE] len=16 with irq_en=0 finished, found by polling
UVM_INFO closure_tb.sv(46) @ 1855000: uvm_test_top [CLOSE] 20-word job finished with DONE
UVM_INFO closure_tb.sv(53) @ 2135000: uvm_test_top [CLOSE] 3-word packet for LEN=8 ended the job with ERR
UVM_INFO dma_tb.sv(1530) @ 2235000: uvm_test_top.env.cov [COV] plan items hit: 13 of 17
UVM_INFO dma_tb.sv(1531) @ 2235000: uvm_test_top.env.cov [COV] plan holes: len=0 len=1 len=2-4 gap=0
UVM_INFO dma_tb.sv(411) @ 2235000: uvm_test_top.env.sb [SB] 39 words matched, 0 mismatched
UVM_INFO dma_tb.sv(1639) @ 2235000: uvm_test_top.env [CMP] in-order comparator: 39 matches, 0 mismatches
UVM_ERROR :    0
UVM_FATAL :    0

Six holes closed in one test, and both checkers agree on all 39 words (16 + 20 + 3). The short packet needed no fault injection, because the DMA's error is defined by the packet being shorter than LEN, which a directed sequence can produce. The bench already has a fault-injection route too: dma_truncate_test swaps in axis_truncating_driver by factory override and gets ERR. It runs on the plain dma_env, though, which has no coverage component, so in this regression its error path is exercised but never counted. A test that is not measured cannot close a hole.

gap=0: a driver limitation

The sampling lesson showed why: axis_driver drops TVALID after every beat and waits for another edge, so two beats are never on consecutive clocks. No seed, constraint or directed sequence can change that; the driver must. A scratch variant keeps TVALID high when the next beat is already waiting and asks for no idle clocks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

// snippet: scratch driver variant (not in the course bench)
class axis_b2b_driver extends axis_driver;
  `uvm_component_utils(axis_b2b_driver)
  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction
  task run_phase(uvm_phase phase);
    bit have, on;
    vif.tvalid <= 1'b0;
    @(posedge vif.rst_n);
    forever begin
      if (!have) seq_item_port.get_next_item(req);
      have = 0;
      if (!on) begin                          // first beat of a burst
        repeat (req.idle) @(posedge vif.clk);
        @(posedge vif.clk);
      end else if (req.idle != 0) begin       // a requested gap
        vif.tvalid <= 1'b0;
        repeat (req.idle) @(posedge vif.clk);
      end
      vif.tdata  <= req.data;
      vif.tlast  <= req.last;
      vif.tvalid <= 1'b1;
      on = 1;
      do @(posedge vif.clk); while (!vif.tready);
      seq_item_port.item_done();
      seq_item_port.try_next_item(req);       // is the next beat already waiting?
      if (req == null) begin
        vif.tvalid <= 1'b0;
        on = 0;
      end else have = 1;
    end
  endtask
endclass

try_next_item returns at once with the next item if the sequence has one ready (it gives the sequence a few delta cycles to arbitrate), so a beat with idle == 0 goes out on the very next clock. When nothing is waiting, TVALID drops, so the driver never presents a stale beat twice. A test that is dma_check_test plus a factory type override of axis_driver with this class:

+UVM_TESTNAME=dma_b2b_test   (scratch run)
UVM_INFO dma_tb.sv(1530) @ 1195000: uvm_test_top.env.cov [COV] plan items hit: 10 of 17
UVM_INFO dma_tb.sv(1531) @ 1195000: uvm_test_top.env.cov [COV] plan holes: len=0 len=16 len>16 irq_en=0 outcome=err clear=err tlast=early
UVM_INFO dma_tb.sv(411) @ 1195000: uvm_test_top.env.sb [SB] 14 words matched, 0 mismatched
UVM_INFO dma_tb.sv(1639) @ 1195000: uvm_test_top.env [CMP] in-order comparator: 14 matches, 0 mismatches

gap=0 is gone from the holes, the same 14 words still match, and the run is 110 ns shorter (it ends at 1195000 ps instead of 1305000 ps) because the jobs move faster. The DMA now sees back-to-back input, which it had never seen in any test.

Watch it run: the closure plan

Merged with dma_check_test, the two bench tests and the two scratch tests reach 17 of 17. One caution about this course's setup: those are plan-tally numbers. Verilator prints every covergroup at 0.0%, so the cross holes from the crosses lesson, such as IRQ_EN = 0 with a zero-length job, are invisible here. On a commercial simulator the merged covergroup report would still show them open.

What breaks: the hole that hid two constraints

The len>16 story is the one to remember. The plan row looked like a constraint hole with an obvious fix: override len_c. The fix failed with a fatal at 1005 ns, because a second, independent constraint in the packet sequence encoded the same assumption. Every layer of a bench (job config, sequence, driver) carries its own picture of "normal traffic", and a closure test has to override all of them. Expect that, and read the fatal as information: it found a limit nobody had written down.

The irq_en=0 story is the same lesson from the other side. The test helper run_job assumes the interrupt, so relaxing the soft constraint would not have produced a covered hole, it would have produced a hung test. Closure tests often expose assumptions in the test infrastructure, which is a good reason to write them early.

Next: with the holes closed, the remaining question is what the logs say and how a test decides it failed. Reporting and Verbosity Control covers that, including the SB_EMPTY demotion used above.

Related topics

• Up: Coverage & Closure

• Regression merge and the closure loop

• SystemVerilog: closing techniques