Error-handling logic has an odd life. In the field it runs rarely, often only when something else has already gone wrong. In a testbench built from legal traffic it never runs at all. And when it finally does run, in the lab or at a customer, the system is already in trouble, so a bug in the recovery path turns a recoverable error into a hang. The code that matters most when things go wrong is the code tested least.

How do you verify that a design handles bad input correctly?

The obvious answer, and why it breaks

The obvious answer is one directed test per error: send one bad CRC, check the flag. Lesson 4.1's suite did exactly that and missed BUG_13 (no error report for high-priority packets) because its one bad packet was low priority. Errors interact with everything else: priority, port, length, counters at their limits, clearing, and what happens to the next good packet. One error per kind tests one point of a large space.

The idea: inject faults systematically, check every response

The faults come from an explicit error mode, not loose constraints (lesson 4.3): the test decides what is broken and so knows exactly what response to expect.

Build it: error injection on DCRouter

The test sends 40 rounds of one packet per input, each packet a random mix of good, bad-CRC and length-0, at random priority, destination and length. The scoreboard expects out_err on exactly the faulty tails. Then the counters must equal the faults injected per input; 300 more faults must saturate ERR_CNT1 at 255; clearing ERR_CNT0 must leave the others alone; and an unmapped address must answer pslverr.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

// uses: dcrouter
// Lesson 4.7: error injection. Every input sends a mix of good packets, bad-CRC packets and
// length-0 headers, at both priorities and every length, interleaved at random. The
// scoreboard expects out_err on exactly the faulty tails; the error counters must equal the
// faults injected per input. Then: saturation, clearing one counter, and an unmapped address.
module tb;
  `include "dcr_env.svh"
  initial begin
    reset_dut();
    // E1 faults of both kinds, both priorities, every input, mixed with good packets (R5, R8)
    for (int n = 0; n < 40; n++)
      for (int i = 0; i < 4; i++) begin
        int kind;
        kind = $urandom_range(0, 3);                 // 0, 1: good   2: bad CRC   3: length 0
        queue_pkt(i, $urandom_range(0, 3), $urandom_range(0, 1),
                  kind == 3 ? 0 : $urandom_range(1, 15), kind == 2);
      end
    wait_drain(20000);
    // E2 each input's counter equals the faults it injected; good packets do not count (R9)
    for (int k = 0; k < 4; k++) check_reg(8'h10 + 8'(4 * k), bad_pkts[k], $sformatf("ERR_CNT%0d", k));
    // E3 saturation: 300 more faults on input 1 leave ERR_CNT1 at 255 (R9)
    for (int n = 0; n < 300; n++) queue_pkt(1, 0, 0, 1, 1);
    wait_drain(40000);
    check_reg(8'h14, 255, "ERR_CNT1 after 300 more faults");
    // E4 clearing ERR_CNT0 leaves the others alone (R9)
    apb_wr(8'h10, 0);
    check_reg(8'h10, 0, "ERR_CNT0 after its clear");
    check_reg(8'h14, 255, "ERR_CNT1 after clearing ERR_CNT0");
    check_reg(8'h18, bad_pkts[2], "ERR_CNT2 after clearing ERR_CNT0");
    // E5 a fault on the register bus: an unmapped address answers pslverr (R12)
    apb_rd(8'h40);
    if (rd_err !== 1'b1) fail("no pslverr for unmapped address 0x40");
    finish($sformatf("%0d faulty packets among %0d, every one flagged and counted", bad_pkts[0] + bad_pkts[1] + bad_pkts[2] + bad_pkts[3], sent_pkts[0] + sent_pkts[1] + sent_pkts[2] + sent_pkts[3]));
  end
endmodule

It passes on the clean design. Against the Bug Zoo:

clean design: PASS
  BUG_1  caught  ERROR @135: out0: unexpected flit 19b9
  BUG_2  caught  ERROR @145: out2: flit a9d1, expected 38c5 from input 0
  BUG_3  caught  ERROR @1045: out0: out_err 1 on flit 976b, expected 0
  BUG_4  caught  ERROR @195: out1: out_err 0 on flit 0049, expected 1
  BUG_5  missed  
  BUG_6  missed  
  BUG_7  missed  
  BUG_8  caught  ERROR @55: out2: out_err 1 on flit b200, expected 0
  BUG_9  missed  
  BUG_10 missed  
  BUG_11 missed  
  BUG_12 caught  ERROR @975: out1: out_err 1 on flit 0026, expected 0
  BUG_13 caught  ERROR @165: out3: out_err 0 on flit 0062, expected 1
  BUG_14 caught  ERROR @6295: ERR_CNT2: read 28, expected 12
  BUG_15 caught  ERROR @18425: ERR_CNT1 after 300 more faults: read 3d, expected ff
  BUG_16 missed  
  BUG_17 caught  ERROR @200025: 277 flits still pending after 20000 cycles (lost or stuck)
  BUG_18 caught  ERROR @18575: no pslverr for unmapped address 0x40
  BUG_19 caught  ERROR @18515: ERR_CNT1 after clearing ERR_CNT0: read 0, expected ff
  BUG_20 missed  
  BUG_21 missed  
  BUG_22 missed  
  BUG_23 missed  
  BUG_24 caught  ERROR @185: out3: unexpected flit 4101
caught 13 of 24 bugs

How to read it: 13 caught. Compared with lesson 4.1's directed suite it adds BUG_1, BUG_13, BUG_14, BUG_15, BUG_19, BUG_24. BUG_13 (high-priority errors unreported) falls because faults now come at both priorities; BUG_14 (port 2 counts good packets) because every input's counter is compared with its injected faults; BUG_15 (wrap instead of saturate) because the test drives the counter past 255; BUG_19 (one clear clears all) because the other counters are re-read after a clear. The rest of the gain (BUG_1, BUG_24) comes from heavier, more varied traffic.

What error injection should cover

• Every fault type, crossed with modes: priority, port, configuration, length.

• The response and its side effects: flags, counters, interrupts, status, and what is left untouched.

• Limits of the error machinery: counters at saturation, error queues full, back-to-back errors.

• Recovery: the next good transaction after an error must be correct.

• Faults in every interface: data paths, register buses, handshakes (a valid dropped mid-packet), resets mid-transaction.

How real teams inject errors

What breaks

Even this test misses BUG_16 (PKT_CNT3 not cleared): it never clears a packet counter. It also misses three bugs the directed suite caught (BUG_9, BUG_10, BUG_11), because it has no credit audit and never compares PKT_CNT with the packets seen. Error injection was aimed at the error path; credits and packet counters belong to other tests. A plan that lists each rule and the test that drives it (lesson 2.6) shows such gaps before the Bug Zoo does.

Key takeaways

• Error handling runs rarely and is tested least, so its bugs wait longest.

• Inject faults deliberately through an error mode, crossed with modes, and check every response and side effect.

• Drive the error machinery to its limits and check recovery.

Related topics

• Part 4 overview: Stimulus