Part 10 · Advanced Topics · Intermediate

Debugging Hung Phases

Using objection trace logs, display_objections, and a structured triage flow for the most common end-of-run hangs.

Classify the hang before touching waveforms

A 'hung simulation' can come from different roots: leaked objection, blocked driver handshake, deadlocked sequence fork, or timeout logic never reached. Start with objection state first because it is cheap, deterministic, and often immediately conclusive.

diagram

[CHECK] hang triage classes

  A) objection leak
     root count never returns to zero

  B) objection balanced but phase extension condition never clears
     phase_ready_to_end hold stuck

  C) no objection issue, but run never progresses
     sequence/driver handshake deadlock

  D) test waits forever on event/queue with no timeout
     independent of objections, but often misdiagnosed as objection leak

First-line tools: trace and display

Enable +UVM_OBJECTION_TRACE to log every raise/drop with source and count deltas. Then dump current holders with display APIs at timeout boundaries.

bash

# simulator invocation example
simv +UVM_TESTNAME=stress_case +UVM_OBJECTION_TRACE +UVM_TIMEOUT=2ms,NO

systemverilog

task watchdog(uvm_phase phase);
  #1ms;
  `uvm_warning("HANG", "watchdog fired, printing objection state")
  phase.phase_done.display_objections();
endtask

task run_phase(uvm_phase phase);
  fork watchdog(phase); join_none
  // normal test flow...
endtask

diagram

[UVM] sample trace interpretation

  RAISE src=uvm_test_top.env.agt.drv count=1 total=1 reason="traffic start"
  RAISE src=uvm_test_top.env.scb      count=1 total=2 reason="waiting drain"
  DROP  src=uvm_test_top.env.agt.drv count=1 total=1 reason="traffic done"
  ... no DROP from scoreboard ...

  Diagnosis:
    scoreboard temporary hold never released (phase_ready_to_end path stuck).

Common hang patterns and fixes

diagram

Pattern                              Symptom                              Fix
  ──────────────────────────────────────────────────────────────────────────────────────────
  Missing drop in error branch           total count stays >0                    add matched drop path
  join_none worker never signals done    count >0 or hold flag stuck             explicit completion event
  phase_ready_to_end guard bug           repeated raises, no drop                add waiting flag + clear path
  Scoreboard wait condition incomplete   hold on one empty queue while map busy  include all pending stores
  Sequence blocks on finish_item         no progress, objections still held      inspect driver item_done path
  Driver blocks on get_next_item         no traffic, no drops                    sequence start/connect issues

Case study: stuck scoreboard hold

A common production bug: scoreboard raises in phase_ready_to_end, waits on exp_q empty, but ID map was the active store and never checked. Queue empties immediately, map does not, and drop path never triggers due to wrong condition.

systemverilog

// WRONG
wait (exp_q.size() == 0);

// RIGHT
wait ((exp_q.size() == 0) && (exp_by_id.num() == 0));

Condition bugs often look like random hangs under out-of-order traffic.
Always align hold condition with actual scoreboard architecture.
Unit-test drain condition helper functions where possible.

Deterministic hung-phase playbook

Enable objection trace and set finite timeout with diagnostic output.
On timeout, dump current objection holders and counts.
Identify oldest outstanding source in trace; inspect its raise path and all exits.
Check phase_ready_to_end holds and guard flags for stuck temporary objections.
Verify sequence-driver handshake (`start_item/finish_item` ↔ `get_next_item/item_done`).
Patch with minimal ownership fix; rerun same seed before broad regression.

diagram

[CHECK] do-not-do list

  - Do not force-drop objections from unrelated owner just to unblock run.
  - Do not increase global timeout first; diagnose count ownership first.
  - Do not add huge drain_time as a blanket hang workaround.
  - Do not debug with random seed changes before reproducing with fixed seed.

Key takeaways

Hung runs are usually ownership/accounting defects, not mysterious simulator behavior.
Use +UVM_OBJECTION_TRACE and display_objections as first diagnostics.
phase_ready_to_end holds are a frequent source of hidden leaks.
Fix ownership at source; avoid force-drop bandaids.

Common pitfalls

Escalating timeout without collecting objection-state evidence.
Fixing hang by global drain inflation instead of correcting logic.
Changing seed during debug and losing reproducibility.

Practice this lessonQuestions tagged for this topic in the bank