After Part 5, two Bug Zoo bugs had escaped every testbench, and no pass/fail result hinted at them. This lesson asks the testbench a different question, not "was anything wrong?" but "what did you never do?", and uses the answer to write the stimulus that catches them.
What does DCRouter's coverage reveal, and does closing the holes catch the last bugs?
The obvious next step after Part 5 would be more random traffic or more checkers. Neither helps: the two escapes are never activated by the stimulus, and their checks (the R1 assertion, the counter read-backs) already exist. What is missing is knowledge of which actions the stimulus never takes, and only a model of the actions that matter can provide it.
78 bins, each traced to a rule: the header cross (R3 to R6), fault types by priority and length (R5, R8), FIFO full and credits exhausted (R1, R7), simultaneous tails (R10), sleep, wake by each input alone and credits during sleep (R13), every configuration value (R1, R6, R13), and every register operation: reset-value read, STATUS read, clearing each counter, saturating each error counter, an unmapped access (R9 to R12). Each bin is a named counter declared up front, so a bin never hit is reported as a hole.
// dcr_cov.svh: DCRouter step 8, a functional coverage model. Included inside `module tb` after
// dcr_env.svh and dcr_sva.svh (it uses ctrl_q). Each bin is a named counter: bins are declared
// up front, so a bin never hit is reported as a hole. (A covergroup expresses the same model;
// named counters keep it runnable on every simulator the course uses.)
int cov [string];
function automatic void bin(string name); if (!cov.exists(name)) cov[name] = 0; endfunction
function automatic void hit(string name); if (cov.exists(name)) cov[name]++; else $display("ERROR: undeclared bin %s", name); endfunction
function automatic string len_class(int len);
return len == 0 ? "len0" : len == 1 ? "len1" : len == 15 ? "len15" : "len2-14";
endfunction
initial begin
string lc [4] = '{"len0", "len1", "len2-14", "len15"};
for (int d = 0; d < 4; d++) for (int p = 0; p < 2; p++) foreach (lc[k]) bin($sformatf("hdr dst%0d prio%0d %s", d, p, lc[k]));
foreach (lc[k]) if (k != 0) for (int p = 0; p < 2; p++) bin($sformatf("fault bad CRC prio%0d %s", p, lc[k]));
for (int p = 0; p < 2; p++) bin($sformatf("fault len0 prio%0d", p));
for (int i = 0; i < 4; i++) begin
bin($sformatf("input%0d FIFO full", i)); bin($sformatf("output%0d out of credits", i));
bin($sformatf("wake by input%0d alone", i)); bin($sformatf("CTRL port%0d disabled", i));
bin($sformatf("ERR_CNT%0d cleared", i)); bin($sformatf("PKT_CNT%0d cleared", i));
bin($sformatf("ERR_CNT%0d saturated", i));
end
bin("two or more tails in one cycle"); bin("back-to-back packets to different outputs");
bin("sleep entered"); bin("credit returned while asleep");
bin("CTRL prio_en 0"); bin("CTRL prio_en 1"); bin("CTRL lp_en 0");
bin("CTRL read before any write"); bin("STATUS read"); bin("unmapped address accessed");
end
bit cv_hdr_next [4] = '{1, 1, 1, 1}, cv_last_tail [4] = '{0, 0, 0, 0}, cv_ctrl_written = 0, cv_slept = 0;
int cv_last_dst [4] = '{-1, -1, -1, -1}, cv_out_prio [4], cv_out_len [4];
bit cv_started [4] = '{0, 0, 0, 0};
always @(posedge clk) if (!rst) begin
for (int i = 0; i < 4; i++) begin
if (in_vld[i] && in_rdy[i]) begin
logic [15:0] d;
d = in_data[16*i +: 16];
if (cv_hdr_next[i]) begin
hit($sformatf("hdr dst%0d prio%0d %s", d[15:14], d[13], len_class(d[12:9])));
if (cv_last_tail[i] && cv_last_dst[i] != d[15:14]) hit("back-to-back packets to different outputs");
cv_last_dst[i] = d[15:14];
end
cv_last_tail[i] = in_last[i];
cv_hdr_next[i] = in_last[i];
end else if (!in_vld[i]) cv_last_tail[i] = 0;
if (in_vld[i] && !in_rdy[i] && !sleeping && ctrl_q[i]) hit($sformatf("input%0d FIFO full", i));
if (occ[i] == 4) hit($sformatf("output%0d out of credits", i));
if (sleeping && in_vld == 4'(1 << i)) hit($sformatf("wake by input%0d alone", i));
end
for (int o = 0; o < 4; o++) if (out_vld[o]) begin
logic [15:0] d;
d = out_data[16*o +: 16];
if (!cv_started[o]) begin cv_started[o] = 1; cv_out_prio[o] = d[13]; cv_out_len[o] = d[12:9]; end
if (out_last[o]) begin
cv_started[o] = 0;
if (out_err[o]) hit(cv_out_len[o] == 0 ? $sformatf("fault len0 prio%0d", cv_out_prio[o])
: $sformatf("fault bad CRC prio%0d %s", cv_out_prio[o], len_class(cv_out_len[o])));
end
end
if ($countones(out_last) >= 2) hit("two or more tails in one cycle");
if (sleeping && !cv_slept) hit("sleep entered");
cv_slept = sleeping;
if (sleeping && out_crd != 0) hit("credit returned while asleep");
if (psel && penable) begin // register operations, from the APB pins
if (!pwrite && paddr == 8'h00 && !cv_ctrl_written) hit("CTRL read before any write");
if (!pwrite && paddr == 8'h04) hit("STATUS read");
if (pslverr) hit("unmapped address accessed");
if (pwrite && paddr == 8'h00) begin
cv_ctrl_written = 1;
hit($sformatf("CTRL prio_en %0d", pwdata[4]));
if (!pwdata[5]) hit("CTRL lp_en 0");
for (int k = 0; k < 4; k++) if (!pwdata[k]) hit($sformatf("CTRL port%0d disabled", k));
end
for (int k = 0; k < 4; k++) begin
if (pwrite && paddr == 8'h10 + 8'(4 * k)) hit($sformatf("ERR_CNT%0d cleared", k));
if (pwrite && paddr == 8'h20 + 8'(4 * k)) hit($sformatf("PKT_CNT%0d cleared", k));
if (!pwrite && paddr == 8'h10 + 8'(4 * k) && prdata[7:0] == 8'hFF) hit($sformatf("ERR_CNT%0d saturated", k));
end
end
end
function automatic void coverage_report();
int total, got;
total = 0; got = 0;
foreach (cov[name]) begin total++; if (cov[name] > 0) got++; end
$display("coverage: %0d of %0d bins hit", got, total);
foreach (cov[name]) if (cov[name] == 0) $display(" hole: %s", name);
endfunctionThe step-8 testbench is lesson 5.10's checked hunt with the model included. A final phase, written from the holes (shown below), is switched off with +define+NO_CLOSE for the first run.
// uses: dcrouter
// DCRouter step 8: lesson 5.10's checked hunt with the coverage model of dcr_cov.svh, then a
// phase written from the coverage holes: read CTRL before anything writes it, clear and re-read
// every counter, saturate an error counter, access an unmapped address, read STATUS, and
// disable each port while offering it a packet. Compile with +define+NO_CLOSE to run without
// that phase and see the holes it was written to fill.
module tb;
`include "dcr_env.svh"
`include "dcr_hunt.svh"
`include "dcr_sva.svh"
`include "dcr_fair.svh"
`include "dcr_cov.svh"
task automatic hot_spot(bit prio_on);
apb_wr(8'h00, {26'b0, 1'b1, prio_on, 4'hF});
gap_pct = '{0, 0, 0, 0};
drain_pct = '{100, 100, 100, 100};
for (int n = 0; n < 12; n++)
for (int i = 0; i < 4; i++) queue_pkt(i, 1, i >= 2, 2);
wait_drain(20000);
endtask
task automatic close_holes();
int a0;
apb_rd(8'h04); // STATUS
apb_rd(8'h40); // unmapped (R12)
if (rd_err !== 1'b1) fail("R12: no pslverr for unmapped address 0x40");
for (int i = 0; i < 4; i++) queue_pkt(i, i, 0, 2, 1); // one fault per input, so every ERR_CNT is non-zero
wait_drain();
for (int k = 0; k < 4; k++) begin // clear each counter; only that one clears (R9, R10)
apb_wr(8'h10 + 8'(4 * k), 0); bad_pkts[k] = 0;
apb_wr(8'h20 + 8'(4 * k), 0); got_pkts[k] = 0;
for (int j = 0; j < 4; j++) begin
check_reg(8'h10 + 8'(4 * j), bad_pkts[j] > 255 ? 255 : bad_pkts[j], $sformatf("ERR_CNT%0d after clearing ERR_CNT%0d", j, k));
check_reg(8'h20 + 8'(4 * j), got_pkts[j], $sformatf("PKT_CNT%0d after clearing PKT_CNT%0d", j, k));
end
end
for (int n = 0; n < 260; n++) queue_pkt(2, 3, 0, 1, 1); // saturate ERR_CNT2 (R9)
wait_drain(20000);
check_reg(8'h18, 255, "ERR_CNT2 after 260 faults");
for (int k = 0; k < 4; k++) begin // a disabled port accepts nothing (R1)
apb_wr(8'h00, 32'h30 | (32'hF & ~(32'h1 << k)));
a0 = sent_pkts[k];
queue_pkt(k, (k + 1) % 4, 0, 2);
repeat (30) @(posedge clk);
if (pending() != 4) fail($sformatf("R1: disabled port %0d accepted flits", k));
apb_wr(8'h00, 32'h3F);
wait_drain();
end
endtask
initial begin
reset_dut();
`ifndef NO_CLOSE
check_reg(8'h00, 32'h3F, "CTRL after reset (R11)");
`endif
hunt();
hot_spot(1);
hot_spot(0);
`ifndef NO_CLOSE
close_holes();
`endif
final_checks();
coverage_report();
finish($sformatf("covered hunt: %0d packets, every check clean", sent_pkts.sum()));
end
endmoduleWith +define+NO_CLOSE the testbench is lesson 5.10's, plus the model. The clean design passes, and the model reports:
coverage: 48 of 78 bins hit
hole: CTRL lp_en 0
hole: CTRL port0 disabled
hole: CTRL port1 disabled
hole: CTRL port2 disabled
hole: CTRL port3 disabled
hole: CTRL read before any write
hole: ERR_CNT0 cleared
hole: ERR_CNT0 saturated
hole: ERR_CNT1 cleared
hole: ERR_CNT1 saturated
hole: ERR_CNT2 cleared
hole: ERR_CNT2 saturated
hole: ERR_CNT3 cleared
hole: ERR_CNT3 saturated
hole: PKT_CNT0 cleared
hole: PKT_CNT1 cleared
hole: PKT_CNT2 cleared
hole: PKT_CNT3 cleared
hole: STATUS read
hole: fault bad CRC prio0 len1
hole: fault bad CRC prio1 len1
hole: fault len0 prio1
hole: hdr dst0 prio0 len0
hole: hdr dst0 prio1 len0
hole: hdr dst1 prio1 len0
hole: hdr dst2 prio1 len0
hole: hdr dst3 prio0 len0
hole: hdr dst3 prio1 len0
hole: unmapped address accessed
hole: wake by input2 alone
PASS: covered hunt: 370 packets, every check cleanHow to read it: 30 holes. Several are the escapes, named outright:
Hole | What the stimulus never did | Bug it hides |
|---|---|---|
CTRL port0-3 disabled | Disable a port | BUG_23 (escaped Parts 4 and 5) |
PKT_CNT0-3 cleared | Clear any packet counter, PKT_CNT3 included | BUG_16 (escaped Parts 4 and 5) |
CTRL read before any write | Read the reset value before reconfiguring | BUG_17 (missed by the random hunts of lessons 4.10 and 5.10) |
ERR_CNT0-3 saturated | Drive a counter to 255 | BUG_15 |
ERR_CNT0-3 cleared | Clear an error counter | BUG_19 |
unmapped address accessed | Access an unmapped address | BUG_18 |
wake by input2 alone | Wake the router from input 2 only | None planted, but a BUG_21 on input 2 would escape |
fault bad CRC ... len1, fault len0 prio1, hdr ... len0 | Rare header and fault combinations | Rare-bin closure (lessons 6.5, 6.7) |
CTRL lp_en 0 | Configure low power off | None planted; a mode never tested |
The closing phase is ordinary directed stimulus, each step aimed at a hole: read CTRL first, read STATUS, access 0x40, inject one fault per input, clear each counter and re-read all of them, saturate ERR_CNT2, and disable each port in turn while offering it a packet. The clean design passes, and the model reports:
coverage: 66 of 78 bins hit
hole: CTRL lp_en 0
hole: ERR_CNT0 saturated
hole: ERR_CNT1 saturated
hole: ERR_CNT3 saturated
hole: fault bad CRC prio1 len1
hole: fault len0 prio1
hole: hdr dst0 prio0 len0
hole: hdr dst0 prio1 len0
hole: hdr dst1 prio1 len0
hole: hdr dst2 prio1 len0
hole: hdr dst3 prio0 len0
hole: hdr dst3 prio1 len0
PASS: covered hunt: 638 packets, every check cleanAgainst the Bug Zoo, the same testbench:
clean design: PASS
BUG_1 caught ERROR @885: out0: unexpected flit 3d43
BUG_2 caught ERROR @165: out1: flit 0000, expected 3b10 from input 3
BUG_3 caught ERROR @4195: out1: out_err 1 on flit 2811, expected 0
BUG_4 caught ERROR @189185: out2: out_err 0 on flit 00e2, expected 1
BUG_5 caught ERROR @59255: R6: input 3 waited while 4 packets from other inputs won output 1
BUG_6 caught ERROR @257265: R6: low-priority packet from input 0 won output 1 over a waiting high-prior
BUG_7 caught ERROR @256185: R6: input 0 waited while 4 packets from other inputs won output 1
BUG_8 caught ERROR @115: out1: out_err 1 on flit 7580, expected 0
BUG_9 caught ERROR @337215: credit audit: 1 flits left output 0 into a stalled sink, expected 4
BUG_10 caught ERROR @235325: credit audit: 3 flits left output 0 into a stalled sink, expected 4
BUG_11 caught ERROR @245615: PKT_CNT1 after clearing PKT_CNT0: read a2, expected a3
BUG_12 caught ERROR @5005: out3: out_err 1 on flit 0037, expected 0
BUG_13 caught ERROR @36755: out2: out_err 0 on flit 0084, expected 1
BUG_14 caught ERROR @245645: ERR_CNT2 after clearing ERR_CNT0: read 4b, expected 5
BUG_15 caught ERROR @257135: ERR_CNT2 after 260 faults: read 4, expected ff
BUG_16 caught ERROR @246635: PKT_CNT3 after clearing PKT_CNT3: read 4d, expected 0
BUG_17 caught ERROR @55: CTRL after reset (R11): read 37, expected 3f
BUG_18 caught ERROR @245315: R12: no pslverr for unmapped address 0x40
BUG_19 caught ERROR @245585: ERR_CNT1 after clearing ERR_CNT0: read 0, expected 2
BUG_20 caught ERROR @36845: R13: fell asleep in the middle of a packet
BUG_21 caught ERROR @40015: R13: in_vld 1000 did not wake the router
BUG_22 caught ERROR @273915: credit audit: 2 flits left output 0 into a stalled sink, expected 4
BUG_23 caught ERROR @257185: R1: in_rdy high on a disabled port
BUG_24 caught ERROR @815: out0: unexpected flit ff82
caught 24 of 24 bugsHow to read it: 12 holes remain, down from 30, and the testbench catches all 24 bugs. The closing phase alone accounts for BUG_15, BUG_16, BUG_17, BUG_18, BUG_19, BUG_23: every one of them was behind a hole in run 1. Across twelve seeds the merged coverage reaches 74 of 78; the four bins still empty in every seed (CTRL lp_en 0 and three counters never saturated) are decided in lesson 6.6.
Bug | Directed (4.1) | Error inj. (4.7) | Back-pressure (4.8) | Random hunt (4.10) | Checked hunt (5.10) | Covered hunt (6.9) |
|---|---|---|---|---|---|---|
BUG_1 | · | caught | caught | caught | caught | caught |
BUG_2 | caught | caught | caught | caught | caught | caught |
BUG_3 | caught | caught | caught | caught | caught | caught |
BUG_4 | caught | caught | · | caught | caught | caught |
BUG_5 | · | · | · | · | caught | caught |
BUG_6 | · | · | · | · | caught | caught |
BUG_7 | · | · | · | · | caught | caught |
BUG_8 | caught | caught | caught | caught | caught | caught |
BUG_9 | caught | · | caught | caught | caught | caught |
BUG_10 | caught | · | caught | caught | caught | caught |
BUG_11 | caught | · | caught | caught | caught | caught |
BUG_12 | caught | caught | caught | caught | caught | caught |
BUG_13 | · | caught | · | caught | caught | caught |
BUG_14 | · | caught | · | caught | caught | caught |
BUG_15 | · | caught | · | · | · | caught |
BUG_16 | · | · | · | · | · | caught |
BUG_17 | caught | caught | caught | · | · | caught |
BUG_18 | caught | caught | · | · | · | caught |
BUG_19 | · | caught | · | · | · | caught |
BUG_20 | · | · | caught | caught | caught | caught |
BUG_21 | · | · | · | caught | caught | caught |
BUG_22 | · | · | caught | caught | caught | caught |
BUG_23 | · | · | · | · | · | caught |
BUG_24 | · | caught | caught | caught | caught | caught |
Read across the columns, the course's argument in one table: directed tests catch the cases someone listed; error injection and timing randomisation add whole bug classes; constrained random finds coincidences; checkers catch what was reached but unseen; coverage reveals what was never done, and directed stimulus written from it catches the rest.
Holes named the escapes: both bugs that survived Parts 4 and 5 sat behind empty bins.
The fix was cheap once known: a few lines of directed stimulus per hole.
Coverage found risks beyond the zoo: wake by input 2 and lp_en 0 have no planted bug, and are untested anyway.
24 of 24 is not done: the zoo is 24 bugs someone thought of; Parts 7 and 9 to 14 are about the ones nobody did.
Practice | Purpose |
|---|---|
Coverage model traced to the plan and reviewed | Holes map to requirements, as here |
Hole reviews that produce directed tests or steering | Each hole closed, excluded with proof or waived |
Bug seeding or mutation alongside coverage | Evidence that hitting a bin also means checking it (lesson 6.8) |
The closing phase would have caught nothing without Part 5's checks: the R1 assertion is what fails when a disabled port is ready, and the counter read-backs are what fail when a clear misbehaves. Coverage told the stimulus where to go; checks decided what was wrong when it got there. Lesson 6.1's warning holds even at 24 of 24.
"CTRL read before any write": the hunt writes CTRL at the start of every round, so the reset value is overwritten before anything reads it.
A bug like BUG_21 on input 2 would escape every test; the hole marks a rule (R13) not exercised for that input, which is a risk whether or not a bug was planted.
A coverage model written from the rules turned two silent escapes into named holes.
A short directed phase written from the holes brought one testbench to 24 of 24.
Coverage steers stimulus; checks still decide what is wrong.