Most engineers write their first tests the same way: read the specification, and for each feature write a test that exercises it and checks the result. Reset values, a packet on every port, the shortest and longest packet, an error, the credits, the registers, sleep. Each test is clear, each failure is easy to debug, and by the end every heading of the specification has a test. It feels complete.
What do directed tests find, and what do they leave behind?
The obvious belief is that a test for every feature verifies every feature. A directed test verifies the case it chose: one length, one priority, one port, one timing. A feature has many cases, and bugs often live in the ones the test did not pick, or in interactions between features that no single feature's test combines. Directed tests are exactly as good as the list of cases someone thought to write.
Precise: each test targets one rule, so a failure names its cause.
Cheap to debug: short, deterministic, readable.
Reaches chosen states: a sequence can be built to reach a deep state on purpose (lesson 3.1).
Documents intent: the test list is a readable statement of what was checked.
Limited by imagination: it finds only bugs in cases someone wrote down.
Every Part 4 testbench uses one environment, included inside module tb. It drives packets on each input with valid/ready, models a 4-slot sink with credits on each output, keeps a scoreboard of expected flits per flow (input, output), and offers APB tasks. A test only queues packets, changes timing knobs and reads registers; the scoreboard checks every flit automatically.
// dcr_env.svh: the DCRouter test environment (DCRouter step 4). Included inside `module tb`.
// Sources: queue_pkt() adds a packet to an input's queue; a driver per input sends its flits,
// holding data steady while in_rdy is low, with gap_pct[i] percent idle cycles between flits.
// Header = {dst, prio, len, src[1:0], seq[6:0]}; tail = {8'h00, CRC-8} (inverted when bad).
// Sinks: one 4-slot sink per output frees a slot with drain_pct[o] percent chance per cycle and
// returns a credit for it; more than 4 flits in a sink is a credit overrun.
// Scoreboard: an accepted flit is expected at its packet's output, in order within its flow
// (input, output); out_err is expected on the tail of a packet with a bad CRC or length 0.
// APB: apb_wr, apb_rd, check_reg. wait_drain() waits until every queued flit has arrived.
// End of test: finish(name) prints exactly one PASS or FAIL line.
logic clk = 0;
always #5 clk = ~clk;
logic rst = 1;
logic [3:0] in_vld = 0, in_last = 0, in_rdy, out_vld, out_last, out_err, out_crd = 0;
logic [63:0] in_data = 0, out_data;
logic psel = 0, penable = 0, pwrite = 0, pready, pslverr, sleeping;
logic [7:0] paddr = 0;
logic [31:0] pwdata = 0, prdata;
dcrouter dut (.*);
int errors = 0;
function automatic void fail(string msg);
if (errors < 5) $display("ERROR @%0t: %s", $time, msg);
errors++;
endfunction
function automatic logic [7:0] crc8(input logic [7:0] c, input logic [15:0] d);
logic [7:0] x;
x = c;
for (int k = 15; k >= 0; k--) x = (x[7] ^ d[k]) ? {x[6:0], 1'b0} ^ 8'h07 : {x[6:0], 1'b0};
return x;
endfunction
// ---------------- sources ----------------
typedef struct packed { logic last, err; logic [1:0] dst; logic [15:0] data; } flit_t;
flit_t fq [4][$]; // flits waiting to be driven, per input
flit_t drv [4]; // the flit on in_data now
int gap_pct [4] = '{0, 0, 0, 0};
int seq [4] = '{0, 0, 0, 0};
int sent_pkts [4] = '{0, 0, 0, 0}, bad_pkts [4] = '{0, 0, 0, 0};
task automatic queue_pkt(int i, int dst, int prio, int len, bit bad_crc = 0);
logic [15:0] f [$];
logic [7:0] c;
f.push_back({2'(dst), 1'(prio), 4'(len), 2'(i), 7'(seq[i])});
seq[i]++;
for (int k = 1; k <= len; k++) f.push_back(16'($urandom));
c = 8'h00;
foreach (f[k]) c = crc8(c, f[k]);
f.push_back({8'h00, bad_crc ? ~c : c});
foreach (f[k]) fq[i].push_back({k == f.size() - 1, bad_crc || len == 0, 2'(dst), f[k]});
sent_pkts[i]++;
if (bad_crc || len == 0) bad_pkts[i]++;
endtask
logic [16:0] exp_q [4][4][$]; // expected {err on tail, data} per (input, output)
for (genvar i = 0; i < 4; i++) begin : g_src
always @(posedge clk) if (!rst) begin
if (in_vld[i] && in_rdy[i]) exp_q[i][drv[i].dst].push_back({drv[i].last && drv[i].err, drv[i].data});
if (!(in_vld[i] && !in_rdy[i])) begin // never change valid data while stalled
in_vld[i] <= 0;
if (fq[i].size() > 0 && $urandom_range(0, 99) >= gap_pct[i]) begin
drv[i] = fq[i].pop_front();
in_vld[i] <= 1;
in_data[16*i +: 16] <= drv[i].data;
in_last[i] <= drv[i].last;
end
end
end
end
// ---------------- sinks and scoreboard ----------------
int drain_pct [4] = '{100, 100, 100, 100};
int occ [4] = '{0, 0, 0, 0}, got_pkts [4] = '{0, 0, 0, 0}, got_flits [4] = '{0, 0, 0, 0};
int cur_src [4];
bit in_pkt [4] = '{0, 0, 0, 0};
for (genvar o = 0; o < 4; o++) begin : g_snk
always @(posedge clk) begin
int n;
out_crd[o] <= 0;
n = occ[o];
if (!rst && out_vld[o]) begin
logic [15:0] d;
logic [16:0] e;
d = out_data[16*o +: 16];
n++; got_flits[o]++;
if (!in_pkt[o]) begin cur_src[o] = int'(d[8:7]); in_pkt[o] = 1; end
if (exp_q[cur_src[o]][o].size() == 0) fail($sformatf("out%0d: unexpected flit %h", o, d));
else begin
e = exp_q[cur_src[o]][o].pop_front();
if (e[15:0] !== d) fail($sformatf("out%0d: flit %h, expected %h from input %0d", o, d, e[15:0], cur_src[o]));
if (out_err[o] !== (out_last[o] && e[16])) fail($sformatf("out%0d: out_err %0b on flit %h, expected %0b", o, out_err[o], d, out_last[o] && e[16]));
end
if (out_last[o]) begin in_pkt[o] = 0; got_pkts[o]++; end
end
if (n > 4) fail($sformatf("out%0d: %0d flits in a 4-slot sink (credit overrun)", o, n));
if (n > 0 && $urandom_range(0, 99) < drain_pct[o]) begin n--; out_crd[o] <= 1; end
occ[o] = n;
end
end
function automatic int pending();
int n;
n = 0;
for (int i = 0; i < 4; i++) begin
n += fq[i].size() + (in_vld[i] ? 1 : 0);
for (int o = 0; o < 4; o++) n += exp_q[i][o].size();
end
return n;
endfunction
task automatic wait_drain(int max_cycles = 5000);
int c;
c = 0;
while (pending() > 0 && c < max_cycles) begin @(posedge clk); c++; end
if (pending() > 0) fail($sformatf("%0d flits still pending after %0d cycles (lost or stuck)", pending(), max_cycles));
repeat (4) @(posedge clk);
endtask
// ---------------- APB ----------------
logic [31:0] rd;
logic rd_err;
task automatic apb(input bit wr, input logic [7:0] a, input logic [31:0] wd);
@(posedge clk); psel <= 1; penable <= 0; pwrite <= wr; paddr <= a; pwdata <= wd;
@(posedge clk); penable <= 1;
@(posedge clk); rd = prdata; rd_err = pslverr; psel <= 0; penable <= 0; pwrite <= 0;
endtask
task automatic apb_wr(input logic [7:0] a, input logic [31:0] wd); apb(1, a, wd); endtask
task automatic apb_rd(input logic [7:0] a); apb(0, a, 0); endtask
task automatic check_reg(input logic [7:0] a, input logic [31:0] expv, input string what);
apb_rd(a);
if (rd !== expv) fail($sformatf("%s: read %0h, expected %0h", what, rd, expv));
endtask
task automatic reset_dut();
rst <= 1;
repeat (3) @(posedge clk);
rst <= 0;
endtask
task automatic finish(string name);
if (errors == 0) $display("PASS: %s", name);
else $display("FAIL: %s, %0d errors", name, errors);
$finish;
endtask
initial begin #5000000; $display("FAIL: global timeout"); $finish; end
`ifdef WAVE
initial begin $dumpfile("wave.vcd"); $dumpvars(0, tb); end
wire in0_vld = in_vld[0], in0_rdy = in_rdy[0], in0_last = in_last[0];
wire [15:0] in0_data = in_data[15:0];
wire out1_vld = out_vld[1], out1_last = out_last[1], out1_err = out_err[1], out1_crd = out_crd[1];
wire [15:0] out1_data = out_data[31:16];
`endifThe directed tests, one per heading of the specification:
// uses: dcrouter
// DCRouter step 4: directed tests, one per heading of the specification. Each test drives
// one known case and checks its rule; the environment's scoreboard checks every flit.
module tb;
`include "dcr_env.svh"
int n0;
initial begin
reset_dut();
// D1 reset values (R11)
check_reg(8'h00, 32'h3F, "CTRL after reset");
// D2 one packet from every input to the next output (R3)
for (int i = 0; i < 4; i++) queue_pkt(i, (i + 1) % 4, 0, 2);
wait_drain();
// D3 the shortest and the longest packet (R4)
queue_pkt(0, 1, 0, 1);
queue_pkt(0, 1, 0, 15);
wait_drain();
// D4 a length-0 header is flagged and counted (R5)
queue_pkt(0, 2, 0, 0);
wait_drain();
check_reg(8'h10, 1, "ERR_CNT0 after a length-0 packet");
// D5 a bad CRC is flagged and counted (R8, R9)
queue_pkt(1, 0, 0, 3, 1);
wait_drain();
check_reg(8'h14, 1, "ERR_CNT1 after a bad CRC");
// D6 credits: with output 3's sink full, exactly 4 flits leave (R7)
drain_pct[3] = 0;
n0 = got_flits[3];
queue_pkt(0, 3, 0, 8);
repeat (40) @(posedge clk);
if (got_flits[3] - n0 != 4) fail($sformatf("%0d flits left output 3 with no credits returned, expected 4", got_flits[3] - n0));
drain_pct[3] = 100;
wait_drain();
// D7 counters: ERR_CNT0 clears; PKT_CNT matches the packets seen (R9, R10)
apb_wr(8'h10, 0);
check_reg(8'h10, 0, "ERR_CNT0 after a clear");
for (int k = 0; k < 4; k++) check_reg(8'h20 + 8'(4 * k), got_pkts[k], $sformatf("PKT_CNT%0d", k));
// D8 an unmapped address answers pslverr (R12)
apb_rd(8'h40);
if (rd_err !== 1'b1) fail("no pslverr for unmapped address 0x40");
// D9 sleep after idle, then wake with a packet (R13)
repeat (20) @(posedge clk);
if (!sleeping) fail("not sleeping after 20 idle cycles");
queue_pkt(0, 1, 0, 2);
wait_drain();
finish("directed tests D1-D9 all passed");
end
endmoduleHow to read the waveform: in0_data shows header 4400 (destination 1, length 2) and its flits entering input 0; two cycles later the same flits leave output 1 with out1_last on the tail and out1_err low. out1_crd pulses one cycle after each flit, because the sink drains at once. The second packet, header 4201, is D3's length-1 case.
The testbench passes on the clean design. Run with each of the 24 bugs switched on in turn (node scripts/tools/bug-zoo.mjs), it reports:
clean design: PASS
BUG_1 missed
BUG_2 caught ERROR @135: out0: unexpected flit 0000
BUG_3 caught ERROR @245: out1: out_err 1 on flit ab75, expected 0
BUG_4 caught ERROR @485: out2: out_err 0 on flit 00bf, expected 1
BUG_5 missed
BUG_6 missed
BUG_7 missed
BUG_8 caught ERROR @85: out0: out_err 1 on flit 0580, expected 0
BUG_9 caught ERROR @1355: 1 flits left output 3 with no credits returned, expected 4
BUG_10 caught ERROR @1125: 3 flits left output 3 with no credits returned, expected 4
BUG_11 caught ERROR @1365: PKT_CNT1: read 2, expected 3
BUG_12 caught ERROR @215: out1: out_err 1 on flit 004a, expected 0
BUG_13 missed
BUG_14 missed
BUG_15 missed
BUG_16 missed
BUG_17 caught ERROR @55: CTRL after reset: read 37, expected 3f
BUG_18 caught ERROR @1455: no pslverr for unmapped address 0x40
BUG_19 missed
BUG_20 missed
BUG_21 missed
BUG_22 missed
BUG_23 missed
BUG_24 missed
caught 10 of 24 bugsHow to read it: ten bugs caught, fourteen missed. Seven of the catches are the bugs the tests aimed at (reset value, lengths 1, 15 and 0, credits, unmapped address). Three are bonuses: D2's four simultaneous packets happened to exercise simultaneous push and pop (BUG_2), lock release (BUG_8) and simultaneous tails (BUG_11).
Missed | Bug | The directed test chose | The bug lived in |
|---|---|---|---|
BUG_13 | No out_err for high-priority packets | A bad CRC on a low-priority packet | The other priority |
BUG_21 | Port 3 cannot wake the router | Wake from input 0 | The other port |
BUG_19 | Clearing ERR_CNT0 clears all four | Clear ERR_CNT0, read ERR_CNT0 | The counters it did not re-read |
BUG_24 | Back-to-back packets reuse the old destination | D3's two packets, both to output 1 | Consecutive packets to different outputs |
BUG_1 | FIFO full one entry late | Fast sinks, so no FIFO ever filled | A state never reached |
BUG_5, 6, 7 | Arbitration and priority | No test of contention | A feature without a heading |
BUG_14, 15, 16 | Counter details | One read per counter | Other ports, saturation, clearing PKT_CNT3 |
BUG_20, 22, 23 | Sleep mid-packet, credits while asleep, disabled port | Plain sleep and wake; all ports enabled | Combinations of features |
The pattern is the lesson: a directed test is a single point in a feature's space. Two of the misses (BUG_13, BUG_21) were one value away from a test that existed.
Use | Why |
|---|---|
Bring-up and smoke tests | Simple, deterministic, quick to debug when nothing works yet |
Specific deep states | Build the sequence on purpose instead of waiting for chance (lesson 3.1) |
Regression for every fixed bug | The exact failing case stays covered forever |
Corners random will not reach | Lesson 4.5's long tail |
Register and reset checks | Known values, known addresses |
As one layer, not the strategy: a small directed suite for bring-up and known corners, and constrained random (lessons 4.3 to 4.10) for the cases nobody listed. The environment above is reused unchanged by every later test in this part; only the stimulus changes.
Lesson 4.10 runs constrained-random traffic through the same environment and catches 15 bugs, including BUG_13 and BUG_21, which the directed suite missed by one value, and BUG_24. It also misses bugs the directed suite caught, which is why teams keep both.
BUG_2, BUG_8 and BUG_11: D2 sent four packets at once, which exercised simultaneous push and pop, lock release and simultaneous tails without aiming at them.
Sending the bad-CRC packet with high priority (or one at each priority).
So any bug that corrupts, loses or misroutes a flit fails the run, whichever test happens to trigger it; that is how the bonus catches happened.
A directed test checks one known case with one known answer: precise and cheap to debug.
It is a single point in a feature's space; bugs one value away are missed.
Use directed tests for bring-up, deep states, registers and fixed bugs, with random stimulus for the space between.