Every lesson in this part has added one idea about stimulus: directed cases, random breadth, constraints, distributions, layers, faults, timing, seeds. This lesson puts them into one constrained-random test, runs it against the Bug Zoo, and lays all four Part 4 testbenches side by side. The result is the first column-set of the course's bug-catch matrix: measured evidence of which stimulus finds which kind of bug.

What does a good constrained-random test find on its own, and what does it still miss?

The obvious answer, and why it breaks

The obvious expectation is that a rich enough random test finds everything: it has random packets, faults, timing and configuration, so every bug must eventually show up. Stimulus only creates the chance. A bug is found if the stimulus reaches its state, the wrong value propagates, and a check detects it (lesson 3.3). Random stimulus helps with the first condition only, and only for states it can reach by chance.

Build it: the random hunt

Packets are objects with constraints (lesson 4.3): lengths weighted toward the boundaries with a mixture (lesson 4.4), 5% bad CRCs and 2% length-0 headers. Each of 12 rounds picks which inputs are active, every source's idle profile and every sink's drain profile (lesson 4.8), and whether priority is enabled; a random quiet gap after each round lets DCRouter sleep (lesson 4.6). The checks are the environment's scoreboard, the counters and the credit audit.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

// uses: dcrouter
// DCRouter step 5: a constrained-random hunt. Packets are objects with random fields under
// constraints (lengths biased to the boundaries, 5% bad CRCs). Each of 12 rounds also picks
// which inputs are active, every source's idle profile, every sink's drain profile and
// whether priority is enabled; a random quiet gap between rounds lets DCRouter fall asleep.
// Checks: the environment's scoreboard on every flit, then the counters and a credit audit.
module tb;
  `include "dcr_env.svh"
  class packet;
    rand bit [1:0] dst;
    rand bit       prio;
    rand bit [3:0] len;
    rand bit       bad;
    constraint c_len { len dist {0 := 2, 1 := 10, [2:14] :/ 70, 15 := 18}; }   // percent
    constraint c_bad { len != 0 -> bad dist {0 := 95, 1 := 5};
                       len == 0 -> !bad; }                                    // one fault per packet
  endclass
  int gaps [3] = '{0, 40, 92};
  int drains [3] = '{100, 40, 4};
  int n0;
  initial begin
    packet p = new();
    bit [3:0] active;
    reset_dut();
    for (int r = 0; r < 12; r++) begin
      active = 4'($urandom_range(1, 15));
      apb_wr(8'h00, {26'b0, 1'b1, 1'($urandom_range(0, 1)), 4'hF});   // lp_en, random prio_en, all ports on
      for (int k = 0; k < 4; k++) begin
        gap_pct[k] = gaps[$urandom_range(0, 2)];
        drain_pct[k] = drains[$urandom_range(0, 2)];
      end
      for (int n = 0; n < 10; n++)
        for (int i = 0; i < 4; i++)
          if (active[i]) begin
            if (!p.randomize()) fail("randomize failed");
            queue_pkt(i, p.dst, p.prio, p.len, p.bad);
          end
      wait_drain(40000);
      repeat ($urandom_range(0, 30)) @(posedge clk);
    end
    // counters: packets per output, faults per input (saturating at 255) (R9, R10)
    for (int k = 0; k < 4; k++) check_reg(8'h20 + 8'(4 * k), got_pkts[k], $sformatf("PKT_CNT%0d", k));
    for (int k = 0; k < 4; k++) check_reg(8'h10 + 8'(4 * k), bad_pkts[k] > 255 ? 255 : bad_pkts[k], $sformatf("ERR_CNT%0d", k));
    // credit audit, as in lesson 4.8 (R7)
    gap_pct = '{0, 0, 0, 0};
    drain_pct = '{100, 100, 100, 100};
    while (occ.sum() > 0) @(posedge clk);
    repeat (2) @(posedge clk);
    for (int o = 0; o < 4; o++) begin
      drain_pct[o] = 0;
      n0 = got_flits[o];
      queue_pkt(0, o, 0, 15);
      repeat (60) @(posedge clk);
      if (got_flits[o] - n0 != 4) fail($sformatf("credit audit: %0d flits left output %0d into a stalled sink, expected 4", got_flits[o] - n0, o));
      drain_pct[o] = 100;
      wait_drain();
    end
    finish($sformatf("random hunt: %0d packets (%0d faulty) in 12 random rounds, all checks clean", sent_pkts.sum(), bad_pkts.sum()));
  end
endmodule

It passes on the clean design. Against the Bug Zoo:

clean design: PASS
  BUG_1  caught  ERROR @855: out0: unexpected flit 3d43
  BUG_2  caught  ERROR @135: out1: flit 0000, expected 3b10 from input 3
  BUG_3  caught  ERROR @4165: out1: out_err 1 on flit 2811, expected 0
  BUG_4  caught  ERROR @189155: out2: out_err 0 on flit 00e2, expected 1
  BUG_5  missed  
  BUG_6  missed  
  BUG_7  missed  
  BUG_8  caught  ERROR @85: out1: out_err 1 on flit 7580, expected 0
  BUG_9  caught  ERROR @310245: credit audit: 1 flits left output 0 into a stalled sink, expected 4
  BUG_10 caught  ERROR @216575: credit audit: 3 flits left output 0 into a stalled sink, expected 4
  BUG_11 caught  ERROR @240345: PKT_CNT3: read 4a, expected 4c
  BUG_12 caught  ERROR @4975: out3: out_err 1 on flit 0037, expected 0
  BUG_13 caught  ERROR @36725: out2: out_err 0 on flit 0084, expected 1
  BUG_14 caught  ERROR @240435: ERR_CNT2: read 32, expected 4
  BUG_15 missed  
  BUG_16 missed  
  BUG_17 missed  
  BUG_18 missed  
  BUG_19 missed  
  BUG_20 caught  ERROR @36855: out3: unexpected flit e8d3
  BUG_21 caught  ERROR @439965: 92 flits still pending after 40000 cycles (lost or stuck)
  BUG_22 caught  ERROR @255105: credit audit: 2 flits left output 0 into a stalled sink, expected 4
  BUG_23 missed  
  BUG_24 caught  ERROR @785: out0: unexpected flit ff82
caught 15 of 24 bugs

The bug-catch matrix after Part 4

How to read it: the four tests together catch 19 of 24. The random hunt alone catches 15, the most of any single test, and is the only one to catch BUG_21. It catches the concurrency bugs (2, 11, 24), the low-power bugs (20, 21, 22) and the error-path bugs whose trigger is a fault at random priority (13, 14). The directed and error-injection tests catch bugs that need a specific deep step random never takes: reading CTRL before anything rewrites it (17), reading pslverr on an unmapped address (18), saturating a counter (15), clearing one counter and re-reading the others (19).

Why 5 bugs escape every test

Two different gaps. BUG_5, 6 and 7 are reached and not detected: hot spots and priority traffic happen constantly in the random hunt, but every check is about data, and these bugs only change which legal order the packets arrive in. BUG_16 and BUG_23 are never reached: no stimulus clears PKT_CNT3 or disables a port. The first gap is closed by better checks (Part 5), the second by measuring what stimulus reached and steering it (Part 6).

What the matrix teaches

• No single stimulus wins: the best test catches 15; the four together catch 19.

• Random finds coincidences and combinations: concurrency, low power and fault-at-random-priority bugs.

• Directed finds deep, specific steps: register operations, saturation, unmapped addresses.

• Stimulus cannot find what no check sees: fairness and priority bugs need checkers, not more traffic.

• What the stimulus never does is invisible without coverage: nothing reported that no test disabled a port.

How real teams use a result like this

What breaks

Lesson 4.9 showed the random hunt catching BUG_21 with 16 of 20 seeds. The matrix above is one seed. Its random column is a sample; the directed column is not. In a real project the random column would be a regression of many seeds with merged coverage, and the matrix would report the fraction of seeds that catch each bug.

Key takeaways

• One constrained-random test caught 15 of 24 bugs; all four Part 4 tests together caught 19.

• Random finds coincidences and combinations; directed finds deep specific steps.

• The bugs left are reached-but-unchecked (fairness, priority) or never reached (port disable, a register clear): Parts 5 and 6.

Related topics

• Part 4 overview: Stimulus

• DV interview puzzles: Randomization and Constraints