After a cleanup of the base test, the whole regression runs in half the time. Most tests still pass. A few fail with SB_EMPTY at 155 ns, a time at which the DMA has barely been programmed. The engineer who did the cleanup moved a wait into a helper that now runs in the background. Nothing hangs, nothing mismatches: the tests simply stop before the work they were written to check. How do you tell a test that ended early from one that passed quickly, and how do you end a test at the right moment?
The run phase has no idea what your test is doing. It ends when the run-phase objection count reaches zero and the drain time (if any) has passed. Threads started with fork ... join_none are not counted: if every objection drops, the phase ends while they are still running, and check_phase judges a half-finished scenario. Think of the last person out turning off the lights because the sign-in sheet says everyone has left, while someone is still working in a back room. A test with no objection raised at time 0 is the extreme case: it ends at time 0.
This is not a hang, and the bench catches it only because the scoreboard refuses a run with zero compares. Without that guard, an early end is a false pass.
// snippet: from the course DMA bench
// Early-end playbook: the packet is started in the background and the test
// drops its objection at once. The run ends before the data moves.
class dma_early_end_test extends dma_base_test;
`uvm_component_utils(dma_early_end_test)
function new(string name, uvm_component parent);
super.new(name, parent);
install_verdict_server();
endfunction
task run_phase(uvm_phase phase);
axis_pkt_seq pkt = axis_pkt_seq::type_id::create("pkt");
phase.raise_objection(this);
reg_write(KEY, 32'h0000_1111);
reg_write(LEN, 8);
reg_write(CTRL, 32'h3);
if (!pkt.randomize() with { n == 8; }) `uvm_fatal("RAND", "axis_pkt_seq randomize failed")
fork
pkt.start(env.in_agt.sqr);
join_none
phase.drop_objection(this); // BUG: nothing waits for the packet
endtask
endclassA scratch run of the same test with the objection trace on shows the whole story in three lines: raised at 0, dropped at 155 ns, the moment the CTRL write completed, and the scoreboard's guard firing at the same time stamp.
scratch run: +UVM_TESTNAME=dbg_early_trace_test (dma_early_end_test with the objection trace on)
UVM_INFO @ 0: run_objection [OBJTN_TRC] Object uvm_test_top raised 1 run_objection objection(s): count=1 total=1
UVM_INFO @ 155000: run_objection [OBJTN_TRC] Object uvm_test_top dropped 1 run_objection objection(s): count=0 total=0
UVM_INFO @ 155000: run_objection [OBJTN_TRC] Object uvm_test_top all_dropped 1 run_objection objection(s): count=0 total=0
UVM_ERROR dma_tb.sv(407) @ 155000: uvm_test_top.env.sb [SB_EMPTY] no words were compared: the test checked nothing
UVM_INFO dma_tb.sv(411) @ 155000: uvm_test_top.env.sb [SB] 0 words matched, 0 mismatched
UVM_ERROR : 1
UVM_FATAL : 0
TEST VERDICT: FAIL (UVM_ERROR=1, UVM_FATAL=0)The time stamp is the clue. 155 ns is when the last register write finished; no stream data has moved, and the DMA has not raised its interrupt. A run that ends at the moment of a stimulus, rather than at the moment of a response, ended early.
There are three correct ways to end this test, in order of preference for this bench.
Run the packet in the foreground (or join the fork), wait for the DONE interrupt, clear STATUS, then allow a short drain for the last output word before dropping. This is what dma_smoke_test does; the scratch fix of the early-end test is the same body with the fork ... join_none removed:
// snippet: scratch fix (dbg_tb.sv), dbg_early_fixed_test run_phase
phase.raise_objection(this);
reg_write(KEY, 32'h0000_1111);
reg_write(LEN, 8);
reg_write(CTRL, 32'h3);
if (!pkt.randomize() with { n == 8; }) `uvm_fatal("RAND", "axis_pkt_seq randomize failed")
pkt.start(env.in_agt.sqr); // foreground: returns after the last beat is accepted
wait (irq_vif.irq === 1'b1); // the DUT says the job is done
reg_write(STATUS, 32'h6);
#100ns; // the last output word drains
phase.drop_objection(this);phase.get_objection().set_drain_time(this, 100ns) keeps the phase open for a fixed time after the count reaches zero. It is right for a known, bounded pipeline delay after the last response. It is wrong as a fix for this bug: no fixed time is the right one for an unknown amount of work, and a drain long enough for the biggest job slows every small one.
When the scoreboard knows how much is still owed, it can keep the phase open itself: dma_drain_test swaps in dma_scoreboard_eot, which re-raises the objection in phase_ready_to_end while words are still expected, and drops when the last one arrives. The owed count must come from what the DUT was asked to do (LEN), not from what the monitor has seen so far, or a missing word is never owed (war story 3). The mechanics and the trade-offs are in End-of-test strategy.
scratch run: +UVM_TESTNAME=dbg_early_fixed_test
UVM_INFO dma_tb.sv(411) @ 575000: uvm_test_top.env.sb [SB] 8 words matched, 0 mismatched
UVM_ERROR : 0
UVM_FATAL : 0
TEST VERDICT: PASS (UVM_ERROR=0, UVM_FATAL=0)
+UVM_TESTNAME=dma_drain_test
UVM_INFO dma_tb.sv(411) @ 375000: uvm_test_top.env.sb [SB] 6 words matched, 0 mismatched
UVM_ERROR : 0A fast test is not suspicious in itself; dma_zero_len_test ends at 325 ns and compares nothing, and that is correct, because a zero-length job moves no data (it demotes SB_EMPTY on the scoreboard only, for that reason). What separates it from the early end is evidence that the scenario completed. Line the three runs up:
Evidence |
|
|
|
|---|---|---|---|
End time | 155 ns | 575 ns | 325 ns |
Words compared / words the DUT was asked for | 0 / 8 | 8 / 8 | 0 / 0 |
The DUT's completion seen (DONE interrupt) before the drop | No | Yes | Yes ( |
Objection drop vs. last bus activity | Drop at the last register write | Drop 100 ns after the last output word | Drop after the DONE check |
Verdict | FAIL (SB_EMPTY) | PASS | PASS |
Build those columns into the bench and the question answers itself: compare the count with what the DUT was asked for, require the DUT's own completion signal before the drop, and print both in the scoreboard's summary. Then an early end fails loudly however the test was written.
The cleanup in the opening hook is a pattern seen on real teams. A base-test helper used to start a sequence and wait for it. Someone wrapped it in fork ... join_none so that tests could "do other things while the data moves". Tests that had their own wait for the interrupt kept passing. Tests that relied on the helper's wait now dropped their objection right after programming the DUT. In this bench, SB_EMPTY caught every one of them; on a bench without that guard they would have passed for weeks. The fix was to make the helper either blocking or explicitly asynchronous, with a handle the caller must wait on, so that "started" and "finished" can no longer be confused.
fork ... join_none, then calls phase.drop_objection(this). Each sequence uses set_automatic_phase_objection(1). Does the run end early?It depends on the starting phase. Automatic phase objection raises on the sequence's starting phase before pre_start and drops after post_start, and a sequence started with start() from a test has no starting phase unless the test calls set_starting_phase(phase) first. Without it the sequences raise nothing, the count reaches zero at the test's drop, and the run ends early. With it, the count stays above zero until the last sequence finishes. The objection trace shows which case you are in.
phase_ready_to_end while expected.size() != 0. A DUT bug drops one input word, so one prediction is never matched. What happens?Every time the count reaches zero, the scoreboard re-raises because a prediction is still pending, so the phase never ends, up to the library's limit on ready-to-end iterations, or until the timeout. The scoreboard needs its own bound: wait for the outstanding words only up to a time limit, then drop and let check_phase report the leftover word.
0 words matched. One is correct and one ended early. What single extra fact tells them apart?What the DUT was asked to do. If the test programmed LEN=0 (as dma_zero_len_test does), zero compares is the right answer. If it programmed LEN=8, zero compares out of eight owed means the run ended before the work was done.
Next: these playbooks debug one failing test. Regression scales them to thousands of runs: seeds and replay, triage by signature, and the dashboards that show when a bench stops catching bugs.