A simulation stops making progress. The waveform has three hundred signals and a hundred thousand cycles. An engineer who scrolls around looking for something odd may find it in an hour or a day. An engineer with a method asks four questions in order and usually finds the cause in minutes, because every hang and stall has the same shape: something is waiting for something else that will never happen.
What is a reliable method for finding the cause of a stall, hang or corruption on a waveform?
The obvious approach is to start where the symptom was reported and look for anything unusual. In a large design, almost everything looks unusual to someone who did not write it, and the symptom is usually far from the cause, in both time and space. Without a method, debug follows curiosity rather than dependencies.
Symptom | First question | Typical cause |
|---|---|---|
Hang | Which handshake is stuck, and what is it waiting for? | Lost credit or response, dependency cycle (Part 9), missing wake |
Stall (slow, not stopped) | Which condition is false most of the time? | Credits too few for the round trip, arbitration starvation, bubbles |
Corruption | At which interface does the data first differ from the model? | Data changed while stalled, wrong ID matching, overflow |
A sender starts with 3 credits. The receiver returns credits in batches worth 4, once it has drained 4 entries. The design hangs; apply the method.
// A stuck valid, for the debug method. A sender starts with 3 credits (its
// configured initial value) and sends whenever it has one. The receiver drains one
// entry per cycle but returns credits in batches: one crd_ret pulse is worth 4
// credits, sent once it has drained 4 entries since its last return. With only 3
// credits in the loop, the receiver never reaches 4: the sender has data (vld high)
// and no credits, the buffer is empty, and 3 drained credits sit in the receiver
// waiting for a fourth that cannot come. The checker confirms the permanent hang.
module tb;
logic clk = 0;
always #5 clk = ~clk;
initial begin $dumpfile("wave.vcd"); $dumpvars(0, tb); end
int cyc = 0;
always @(posedge clk) cyc <= cyc + 1;
localparam int BATCH = 4;
logic vld = 0, crd_ret = 0;
logic [2:0] credits = 3, occ = 0, owed = 0; // owed: drained, not yet returned
logic send, pop;
always_comb begin
send = vld && credits != 0;
pop = occ != 0;
end
always @(posedge clk) begin
if (cyc >= 1) vld <= 1; // the sender always has data
occ <= occ + send - pop;
// Receiver: return a batch of credits once BATCH have been drained
crd_ret <= 0;
if (owed + pop == BATCH) begin crd_ret <= 1; owed <= 0; end
else owed <= owed + pop;
// Sender
credits <= credits - send + (crd_ret ? BATCH : 0);
end
int stuck = 0;
always @(posedge clk) if (vld && !send) stuck++;
initial begin
#250;
if (stuck > 10 && credits == 0 && occ == 0 && owed == 3)
$display("PASS: valid stuck for %0d cycles: no credits, empty buffer, 3 credits owed but held for a batch of 4", stuck);
else $display("FAIL: stuck %0d credits %0d occ %0d owed %0d", stuck, credits, occ, owed);
$finish;
end
endmoduleApplying the method, slot by slot:
Find the stuck handshake: from slot 4 on, vld is high but send is 0, for ever. Slots 1 to 3 show the only three sends.
Which condition is missing? send needs a credit; credits reaches 0 in slot 4 and never recovers.
What drives credits? Sends (down) and crd_ret (up). crd_ret never pulses in the whole trace.
Is the receiver's state right? occ returns to 0 in slot 5: the buffer is empty. owed climbs 1, 2, 3 in slots 3 to 5 and stops. Conservation check: credits (0) + occupancy (0) + owed (3) = 3, the initial credits, so no credit was lost. All three are owed, held back waiting for a batch of 4 that can never form.
Root cause: the batch size (4) exceeds the credits in the loop (3). Fix: return credits when the batch is full or when the buffer drains, or make the batch no larger than the sender's initial credits.
Put the handshake signals of every interface on one view before looking at anything else.
Use conservation rules: credits + occupancy + in flight = constant; requests issued − responses received = outstanding. A broken sum tells you a leak happened, and when.
Find the first divergence, not the most visible one: the cycle a count first goes wrong is where the bug is.
Let assertions and monitors do the searching: a failing assertion (lesson 15.3) is the method's first step done for you.
Aid | Helps with |
|---|---|
Assertions with rule names | Pointing at the cycle and rule that failed |
Transaction views from monitors | Seeing packets and responses, not just bits |
Silicon trace buffers frozen by a timeout | The same method in silicon (lesson 15.9) |
The batching hang (illustrative) appeared only after a configuration change lowered the sender's initial credits from 4 to 3 to save area. The receiver's batch size was a separate parameter in another block. An assertion that the batch size never exceeds the sender's initial credits, added to the configuration checks, prevented it from recurring.
Credits (0) + occupancy (0) + owed (3) still equals the initial 3: all three are held by the receiver.
Return the owed credits when the buffer drains (or after a timeout), even if the batch is not full.
Every hang is something waiting for something that will never happen: follow the wait.
Find the stuck handshake, the missing condition, the state behind it, and the first cycle that state went wrong.
Conservation rules expose leaks; turn each root cause into an assertion.