A stage that drops words only under back-pressure

A valid/ready register stage fails its stream check: got 4, expected 1. Words seem to come out of order, but only in tests where the sink stalls; with the sink always ready, every test passes. The designer's assertions on the stage are quiet. A second stage, which really does lose words, passes a colleague's regression without a single error. Two benches, two wrong answers. How do you tell which bench to trust?

What people usually do, and why it fails

When the bug is in the test taught the method: check the design against the spec at the first wrong cycle, then check the checker. Its examples were wrong expected values: a model that misreads the spec or borrows the design's signals. That habit, reading the scoreboard's logic, misses three other bench bugs, because the scoreboard's logic is fine in all of them. The bug is in when the bench samples, when it drives, or whether the check runs at all.

The signature

The monitor bug is easiest to see at one clock edge. The bench drives ready on the falling edge, so at the rising edge both sides of the handshake are stable:

The first three checks

These are Localise checks on the bench. Each changes only the bench, so a result that moves clears the design:

1. Does the checker run? Print how many comparisons it made, next to how many it should have made, and run it once against a design with a known bug (a mutant). Zero comparisons, or a clean mutant, means it is blind.

2. When does the bench sample and drive? Monitors should sample at the design's edge, before the design's registers update (a clocking block does this); drivers should change inputs away from that edge. Look for # delays in monitors and blocking drives on the clock edge.

3. Does the failure move when only the bench's timing changes? Run the other simulator, move the sample point, or drive on the other edge. If the result moves with the design untouched, the bug is in the bench.

One real reproduction: one stage, four monitors

The bench drives two valid/ready stages with sequence numbers 1 to 60 and random back-pressure. One stage is correct; the other accepts a word while stalled and overwrites the one it holds, a real bug. Four copies of one monitor and in-order checker watch them: correct, sampling just after the edge, and with the check disabled by rst_n where !rst_n was meant.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

// stage: a valid/ready register stage. A word moves when valid && ready at a rising edge.
// DROP = 0 is correct. DROP = 1 is a real design bug: it accepts a new word while stalled and
// overwrites the one it holds, so words are lost under back-pressure.
`timescale 1ns/1ns
module stage #(parameter bit DROP = 0) (
  input  logic       clk, rst_n, in_valid, out_ready,
  input  logic [7:0] in_data,
  output logic       in_ready, out_valid,
  output logic [7:0] out_data
);
  assign in_ready = DROP ? 1'b1 : (!out_valid || out_ready);
  always_ff @(posedge clk or negedge rst_n)
    if (!rst_n) begin out_valid <= 1'b0; out_data <= '0; end
    else if (in_valid && in_ready) begin out_valid <= 1'b1; out_data <= in_data; end
    else if (out_ready) out_valid <= 1'b0;
endmodule

// mon_chk: a monitor and an in-order checker on the stage's output. The words are sequence
// numbers 1, 2, 3, ... so each accepted word must be one more than the last.
//   LATE = 1: bench bug, the monitor samples just after the edge instead of at it.
//   DIS  = 1: bench bug, the check is disabled by "rst_n" where "!rst_n" was meant.
module mon_chk #(parameter bit LATE = 0, parameter bit DIS = 0) (
  input logic       clk, rst_n, valid, ready,
  input logic [7:0] data
);
  logic [7:0] last = 8'd0;
  int         checks = 0, errors = 0;
  string      first = "";
  always @(posedge clk) begin
    if (LATE) #1;                              // values after the edge belong to the next cycle
    if (!(DIS ? rst_n : !rst_n) && valid && ready) begin
      checks++;
      if (data !== last + 8'd1) begin
        errors++;
        if (first == "") first = $sformatf("%0t ns: got %0d, expected %0d", $time, data, last + 8'd1);
      end
      last = data;
    end
  end
endmodule

// src: sends sequence numbers 1 to 60, with gaps. It drives on the falling edge, so nothing the
// design samples changes at the design's own edge.
module src (
  input  logic       clk, rst_n, in_ready,
  output logic       in_valid,
  output logic [7:0] in_data
);
  int unsigned rng = 32'h1234_5678;
  bit took = 1'b0;
  initial begin in_valid = 1'b0; in_data = 8'd0; end
  always @(posedge clk) took = in_valid && in_ready;     // the handshake, sampled at the edge
  always @(negedge clk) if (rst_n) begin
    if (took) in_valid = 1'b0;                           // the word was taken at the last edge
    rng ^= rng << 13; rng ^= rng >> 17; rng ^= rng << 5;
    if (!in_valid && in_data < 8'd60 && (rng % 100) < 70) begin
      in_data = in_data + 8'd1;
      in_valid = 1'b1;
    end
  end
endmodule

$ iverilog -g2012 -o tb.vvp tb_bugs.sv && vvp -n tb.vvp
good design,   monitor at the edge:     60 checks, clean
good design,   monitor after the edge:  53 checks, 5 errors, first at 86 ns: got 4, expected 1
broken design, checker on:              38 checks, 15 errors, first at 105 ns: got 5, expected 4
broken design, checker disabled:        0 checks, clean
PASS: a late monitor blames a good design; a disabled checker passes a broken one with 0 checks
...

Verilator 5.048 prints the same five lines. Read them as a reviewer of the bench, not of the design.

• Line 2 blames a correct design. The late monitor made 53 comparisons instead of 60 and reports got 4, expected 1: pairing each ready with the next cycle's word, it skipped words 1 to 3 and counted others at the wrong time, as in the picture above. Line 1, the same design and checker sampling at the edge, is clean with all 60. Only the bench changed, so the design is cleared.

• Line 3 is a real design bug, and its message has the same shape as line 2: an unexpected number. Shape alone cannot tell a bench bug from a design bug; changing only the bench can.

• Line 4 passes a broken design with 0 checks. rst_n is 1 for the whole test, so a check guarded by rst_n instead of !rst_n never runs. Without its count, this line is indistinguishable from line 1. The bench's own pass rule treats zero comparisons as a failure.

A race between the bench and the design

The third class needs no monitor at all. Here the bench drives the design's input with a blocking assignment on the design's own edge:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// bad-example: a race, kept short so it can be run in both simulators (output below)
// race_tb: the bench drives the DUT's input with a blocking assignment on the DUT's own edge.
// Whether the flop sees the old or the new value depends on which process the simulator runs first.
`timescale 1ns/1ps
module dut (input logic clk, input logic [7:0] d, output logic [7:0] q);
  always_ff @(posedge clk) q <= d;
endmodule

module race_tb;
  logic clk = 1'b0;
  logic [7:0] d = 8'd0, q;
  always #5 clk = ~clk;
  dut u (.clk, .d, .q);
  initial begin
    repeat (4) begin @(posedge clk); d = d + 1; end   // bad: blocking drive on the DUT's edge
    @(negedge clk) $display("q=%0d after 4 drives", q);
    $finish;
  end
endmodule

$ iverilog -g2012 -o race.vvp race.sv && vvp -n race.vvp
q=3 after 4 drives
...
$ verilator --binary --timing -Wno-fatal --top-module race_tb -Mdir vlr race.sv && ./vlr/Vrace_tb
q=4 after 4 drives
...

Same code, two answers, one cycle apart. At each rising edge two processes run: the flop reads d, and the bench writes it. The language does not fix their order. Icarus ran the flop first and Verilator the bench first. Moving the drive to @(negedge clk) gives q=4 in both. A nonblocking drive on the rising edge removes the race in the language: the flop reads d before the update, so the standard answer is q=3, which Icarus prints. Verilator 5.048 still printed 4, a difference in how it schedules a process woken by the clock. So the portable rule is to drive away from the sampling edge, or through a clocking block, and not to rely on any one simulator's ordering.

The fix pattern

• Sampling: sample at the design's edge, before its registers update: in an always @(posedge clk) without delays, or with a clocking block's input skew. Never add a # delay to a monitor to "see the new value".

• Races: drive inputs on the other edge or through a clocking block, and release reset between edges, never on one the bench also acts on.

• Disabled checkers: checks on by default, a disable that is explicit and logged, and conditions written in the same polarity as the reset they name.

Preventing it

• Count every check and fail on zero. Every checker prints its comparison count at the end; the test fails if a checker that should have run made none. The bench above does this.

• Keep a few mutants in the regression, each expected to fail, as Part 3 recommends. A disabled or late checker turns one of them green the day it breaks.

• Run part of the regression in a second simulator. A race shows up as a difference before it shows up as a bug report.

• The review questions: on which edge does each monitor sample and each driver drive, and how many comparisons did this checker make in this test? See UVM's false-pass playbook for the UVM forms (objections, report catchers, disabled checks).

Traps

Related topics

• Part 5 · Bug Classes

• When the bug is in the test: a wrong scoreboard

• UVM: debugging the testbench

• UVM: the mismatch playbook

• SystemVerilog: anatomy of a race

• SystemVerilog: the clocking-block cure

• Part 10: case studies, including the testbench that was wrong