A stream block passes its unit test every night. In the subsystem regression it fails a few seeds a week: the scoreboard says word 2 arrived as 63 when it expected 52. The data path is two lines of code and looks right. The block owner suspects the scoreboard; the scoreboard owner suspects the block.
Nobody has asked the question that matters: did either side break the handshake?
The usual move is to trace 63 back through the data path, find nothing wrong, and blame a race or the bench. The data path is innocent: the word was right but presented at the wrong time. A protocol bug lives in the control of a transfer, so a search that follows the value walks straight past it.
Valid/ready has three rules that almost every bus and stream shares (the RTL course owns them in depth). Each one, broken, leaves its own mark:
Rule broken | What you see in a log | What you see in a waveform |
|---|---|---|
Valid dropped before ready | a missing word or transaction; sometimes a hang if the receiver was half-way through | valid high, ready low, then valid low with no handshake in between |
Payload changed while waiting | a skipped or wrong word, often one that belongs to a later transfer | data moves in a cycle where the previous cycle had valid 1 and ready 0 |
Response to the wrong master | one master gets an extra response, the other waits for ever | rvalid at a master with nothing outstanding |
The regression pattern is just as telling: it passes when the receiver never stalls and fails when it does. Tests with an always-ready sink, a fast slave or a single master never exercise the rules.
Is there a rules checker on this port, and is it on? If not, add one before anything else (the one below is 20 lines). If it is, read its first message, not the scoreboard's.
Find the first stall before the failure. Search back from the failing cycle for a cycle with valid 1 and ready 0. Compare valid and the payload in that cycle and the next. A change there is the bug.
Re-run with the receiver always ready. If the failure disappears, the handshake is the suspect. Then count requests against responses per master: any master with more responses than requests was answered for someone else.
Read it by finding the stall, then asking what the data did during it.
Slot 0: ready is 1, both sources hand over 41.
Slots 1 to 2: the sink stalls. data_good holds 52; data_bad moves to 63 in slot 2, while valid waits. That is the violation.
Slot 3: ready returns. The good source delivers 52; the bad one delivers 74. Words 52 and 63 are gone, and nothing on the data path computed them wrongly.
The source sends 16 words from a table. The bug is one condition: the read pointer moves when valid is high, not when valid && ready. The bench has a scoreboard and a small rules checker; FIX selects the fixed source and is 1 by default, so the bench as shown passes.
// rd_source: sends N words from a small table on a valid/ready port.
// FIX=0 is the bug: the read pointer moves on valid, not on the handshake,
// so while the sink stalls the data changes under a waiting valid.
module rd_source #(parameter bit FIX = 1, parameter int N = 16) (
input logic clk, rst_n,
output logic valid,
input logic ready,
output logic [7:0] data
);
logic [7:0] table_q [N];
logic [4:0] rd_ptr;
initial for (int i = 0; i < N; i++) table_q[i] = 8'(8'h30 + i * 8'h11);
assign valid = (rd_ptr < 5'(N));
assign data = table_q[rd_ptr[3:0]];
always_ff @(posedge clk or negedge rst_n)
if (!rst_n) rd_ptr <= '0;
else if (FIX ? (valid && ready) : valid) rd_ptr <= rd_ptr + 5'd1;
endmodule
// vr_rules: the valid/ready rules for one port, as a bench checker.
// R1 valid and, while valid, data are never X
// R2 once valid is high without ready, valid stays high and data stays stable
module vr_rules #(parameter string NAME = "port") (
input logic clk, rst_n, valid, ready,
input logic [7:0] data
);
logic pend_q = 1'b0; logic [7:0] data_q; int unsigned cyc = 0;
always @(posedge clk) begin
cyc++;
if (rst_n) begin
if ($isunknown(valid) || (valid && $isunknown(data)))
begin $display("FAIL %s R1 cycle=%0d: valid=%b data=%h", NAME, cyc, valid, data); $fatal(1); end
if (pend_q && !valid)
begin $display("FAIL %s R2 cycle=%0d: valid dropped before ready", NAME, cyc); $fatal(1); end
if (pend_q && data !== data_q)
begin $display("FAIL %s R2 cycle=%0d: data changed while waiting: %h -> %h", NAME, cyc, data_q, data); $fatal(1); end
end
pend_q <= rst_n && valid && !ready;
data_q <= data;
end
endmodule
Four runs, each a question:
$ iverilog -g2012 -o pv.vvp pv_tb.sv && vvp -n pv.vvp
PASS: 16 words in order, no rule broken (FIX=1)
...
$ iverilog -g2012 -Ppv_tb.FIX=0 -o pvb.vvp pv_tb.sv
$ vvp -n pvb.vvp +NOSTALL
PASS: 16 words in order, no rule broken (FIX=0)
...
$ vvp -n pvb.vvp +NOCHECK
FAIL scoreboard cycle=6: word #2 = 63, expected 52
...
$ vvp -n pvb.vvp
FAIL src R2 cycle=6: data changed while waiting: 52 -> 63
...
FAIL scoreboard cycle=6: word #2 = 63, expected 52`+NOSTALL`, bug in: PASS. With a sink that is always ready, valid and the handshake are the same thing, so the wrong condition is never wrong. This is how the bug got through its unit test.
`+NOCHECK`: the scoreboard fails at cycle 6 with a message about data: word 2 is 63. Read alone, it sends you into the data path.
Both checks on: in the same cycle the rules checker says data changed while waiting: 52 -> 63. That names the rule, the port and the two values. The fix is now obvious: the pointer must move on valid && ready.
Verilator 5.048 fails at the same cycle but prints the scoreboard line first. When two checks fire in one cycle, read every FAIL line of that cycle: the order is the simulator's choice, not a ranking.
dbglab's bench has the same rules for both bus masters in tb/bus_rules.sv: R1 (no X on valid, address and write), R2 (valid and payload held until ready) and R3 (a response only for an accepted request, one in flight). To see R3 work, a scratch copy of HEAD changes one line in the fabric, the kind of change that looks harmless in review: send the response to whoever is asking, instead of the owner.
// snippet: scratch mutation of dbglab rtl/bus_fabric.sv line 54 (dbglab itself unchanged)
assign m_rvalid = rsp_fire ? (m_valid[1] ? 2'b10 : 2'b01) : 2'b00; // to the master that is asking
// the original line: assign m_rvalid = rsp_fire ? grant : 2'b00; // routed to the owner, never to a requester$ for t in smoke dma_copy periph_stream random; do tb/run.sh $t 1; done
dbglab: test=smoke seed=1
PASS test=smoke seed=1 cycles=76 time=756 ns
dbglab: test=dma_copy seed=1
FAIL bus_rules dma R3 at 245000: rvalid with no request in flight
dbglab: test=periph_stream seed=1
PASS test=periph_stream seed=1 cycles=687 time=6866 ns
dbglab: test=random seed=1
FAIL bus_rules cpu R3 at 605000: rvalid with no request in flightThe two tests with only one master on the bus pass, because the requester and the owner are always the same master. As soon as the CPU and the DMA overlap, the checker fires at 245 ns and names the port that got a response it never asked for. With the checker's $fatal removed in the same copy, dma_copy runs on with the DMA out of step and ends 2,000 cycles later: WATCHDOG at cycle 2028, the CPU's request to 0000200c never accepted. Without R3 this is a hang, and you would start in the hang lessons instead of at the one line that is wrong.
Advance on the handshake. Every pointer, counter and state change that means this transfer happened uses valid && ready, never valid alone.
Hold the payload in a register while waiting. Load it only when the stage is empty or the word is leaving (!valid || ready). A skid buffer does this for you.
Route responses from a registered owner. Capture who was granted at issue and send the response there, never to the current requester or the next grant.
The checker on this page and dbglab's bus_rules are the prevention: they cost 20 lines and fire at the first stall. As concurrent assertions they are two lines each, valid && !ready |=> valid && $stable(payload) and a response count against accepts per master (assertion debugging is owned by SystemVerilog: assertion debug). In review, ask of every pointer or counter in a transfer path: does it move on the handshake or on valid?
R3: a response arrived with nothing outstanding at that master, so one response was sent to the wrong master (or the slave answered twice). Capture the fabric's owner at issue and at the response, and the grant at the response. If the response follows the current grant or requester rather than the owner recorded at issue, the routing is wrong. A silent checker means it has no R3 count; add one.
No. The source still moves its pointer on valid; the change only alters which cycles stall, so this test's pattern no longer exposes it. Run the rules checker with a different stall pattern and it fires again. The fix belongs in the source: advance on valid and ready.
Small design (pv_tb.sv, the bench on this page), 2026-10-18, Icarus Verilog 13.0: iverilog -g2012 -o pv.vvp pv_tb.sv && vvp -n pv.vvp prints PASS (FIX=1). iverilog -g2012 -Ppv_tb.FIX=0 -o pvb.vvp pv_tb.sv, then vvp -n pvb.vvp +NOSTALL (PASS), vvp -n pvb.vvp +NOCHECK (scoreboard FAIL at cycle 6) and vvp -n pvb.vvp (R2 FAIL at cycle 6, then the scoreboard line). Verilator 5.048 (verilator --binary --timing --assert -Wno-fatal -GFIX=0 --top-module pv_tb) fails at the same cycle and prints the scoreboard line first.
dbglab at HEAD 49c5f04 (built with dbglab/build-history.sh): a scratch copy of rtl/ and tb/ with rtl/bus_fabric.sv line 54 changed to route m_rvalid to the requesting master. tb/run.sh dma_copy 1 fails bus_rules dma R3 at 245000, tb/run.sh random 1 (Icarus and Verilator) fails bus_rules cpu R3 at 605000; smoke and periph_stream seed 1 pass. With $fatal removed from tb/bus_rules.sv in that copy, dma_copy 1 ends in a WATCHDOG at cycle 2028.
Waveform: COURSE=dbg node scripts/tools/gen-waves.mjs bug-classes:protocol-violations (expectations matched; a second run changed nothing). Limits: one small source and one fabric mutation, simulation only; dbglab's checker does not check write data for X.