Integration Debug Playbook: Hung Seq, Mirror Fail, No Predictor

Top three failure classes

Most integration incidents fit one of three classes: sequence hangs, mirror mismatches, or missing prediction flow. Use a decision tree and targeted probes instead of random edits.

[UVM][RAL] failure classifier

A) Hung register sequence
   Symptom: reg.write/read never returns

B) Mirror mismatch
   Symptom: mirror(UVM_CHECK) fails although DUT appears correct

C) No predictor effect
   Symptom: monitor sees bus traffic but mirror never updates

First 5-minute triage checklist

1. Capture one failing register and one exact timestamped failure log.

2. Check map set_sequencer binding and adapter instance identity.

3. Check auto_predict mode and predictor wiring status.

4. Check monitor transaction completeness (not half-cycle samples).

5. Check whether multiple reg_model handles exist in hierarchy.

• Avoid changing sequences first; verify infrastructure before stimulus logic.

• Collect evidence with small probes to avoid masking race conditions.

• Fix one failure class fully before touching secondary symptoms.

Playbook A: Hung register sequence

Likely causes

• default_map has no sequencer bound via set_sequencer.

• Bound sequencer exists but driver is not running or not connected.

• Adapter reg2bus creates malformed item causing driver stall.

• Bus protocol wait condition (ready/valid) never satisfied.

Probe sequence

1. Print map sequencer full name right before reg.write().

2. Add driver heartbeat log at item start and item done.

3. Print adapter.reg2bus output for the hanging operation.

4. Check interface handshake signals around hang window.

// Probe 1: pre-write guard
uvm_sequencer_base sqr_h;
sqr_h = env.reg_model.default_map.get_sequencer();
if (sqr_h == null)
  `uvm_fatal("RAL_HANG", "default_map sequencer is null")
else
  `uvm_info("RAL_HANG", {"default_map sequencer=", sqr_h.get_full_name()}, UVM_LOW)

// Probe 2: adapter print
virtual function uvm_sequence_item reg2bus(const ref uvm_reg_bus_op rw);
  apb_item item = apb_item::type_id::create("item");
  item.write = (rw.kind == UVM_WRITE);
  item.addr  = base_addr + rw.addr;
  item.data  = rw.data;
  `uvm_info("RAL_HANG", $sformatf("reg2bus kind=%0d addr=0x%0h data=0x%0h",
                                  rw.kind, item.addr, item.data), UVM_LOW)
  return item;
endfunction

Hung sequence quick decision
---------------------------
get_sequencer() == null    -> missing set_sequencer
driver no heartbeat        -> sequencer-driver binding issue
driver heartbeat no bus    -> protocol interface or vif issue
bus active no completion   -> monitor/driver completion condition issue

Playbook B: Mirror mismatch

Likely causes

• Predictor not connected to monitor.ap.

• auto_predict disabled with no explicit predictor path.

• Adapter bus2reg address translation incorrect (base mismatch).

• Model access policy mismatch (W1C/RC/volatile not modeled correctly).

• Backdoor or external master changed register without prediction.

Probe sequence

1. Print auto_predict mode and predictor connection count.

2. Log one monitor item and corresponding bus2reg conversion.

3. Compare mapped offset with expected register offset.

4. Read register frontdoor and print get_mirrored_value() side-by-side.

uvm_status_e   st;
uvm_reg_data_t rd;
uvm_reg_data_t mir;

env.reg_model.ctrl.read(st, rd, UVM_FRONTDOOR, .parent(this));
mir = env.reg_model.ctrl.get_mirrored_value();
`uvm_info("RAL_MIRROR", $sformatf("read=0x%0h mirror=0x%0h", rd, mir), UVM_LOW)

env.reg_model.ctrl.mirror(st, UVM_CHECK, UVM_FRONTDOOR, .parent(this));
if (st != UVM_IS_OK)
  `uvm_error("RAL_MIRROR", "mirror check failed")

[UVM][RAL] mirror mismatch decision tree

mirror check fails
   │
   ├─ predictor connected? ---- no ----> connect monitor.ap -> predictor.bus_in
   │
   ├─ auto_predict=0 and predictor absent? ---- yes ----> wire predictor
   │
   ├─ bus2reg offset correct? ---- no ----> fix adapter.base_addr translation
   │
   ├─ external/backdoor writes present? ---- yes ----> add explicit predict path
   │
   └─ policy mismatch (W1C/RC/volatile) ----> fix model field policies

Playbook C: No predictor updates

Likely causes

• monitor.ap not connected (or connected to wrong export).

• Monitor emits items before transaction is complete.

• bus2reg cast fails due to wrong item type.

• predictor.map not assigned or assigned to wrong map.

Probe sequence

1. Count monitor writes and predictor write callbacks.

2. Add cast failure counters in adapter.bus2reg.

3. Print predictor.map.get_name() at start_of_simulation.

4. Inject one known bus write and inspect predict() activity.

virtual function void bus2reg(uvm_sequence_item bus_item, ref uvm_reg_bus_op rw);
  apb_item item;
  if (!$cast(item, bus_item)) begin
    `uvm_error("RAL_PRED", "bus2reg cast failed")
    rw.status = UVM_NOT_OK;
    return;
  end
  rw.kind   = item.write ? UVM_WRITE : UVM_READ;
  rw.addr   = item.addr - base_addr;
  rw.data   = item.data;
  rw.status = item.slverr ? UVM_NOT_OK : UVM_IS_OK;
  `uvm_info("RAL_PRED", $sformatf("predict kind=%0d addr=0x%0h data=0x%0h",
                                   rw.kind, rw.addr, rw.data), UVM_LOW)
endfunction

[UVM][RAL] predictor observability counters

counter_mon_items       // increment in monitor write()
counter_pred_items      // increment in predictor write()
counter_bus2reg_fail    // increment on cast failure
counter_predict_ok       // increment when rw.status == UVM_IS_OK

Interpretation:
mon>0 pred=0      -> connection issue
pred>0 predict=0  -> adapter conversion issue
predict>0 drift   -> model/policy or competing write issue

Stabilization strategy after root fix

1. Re-run minimal smoke write/read/mirror test with high logs.

2. Run uvm_reg_hw_reset_seq to validate baseline model correctness.

3. Run one constrained random register sequence.

4. Lower debug verbosity once counters and checks are stable.

Key takeaways

• Debug by failure class: hung sequence, mirror mismatch, or no predictor flow.

• Use targeted probes on map binding, monitor feed, and adapter conversion.

• Stabilize with smoke then reset sequence before broad regressions.

Common pitfalls

• Changing multiple layers at once and losing causal evidence.

• Treating monitor traffic as proof of prediction without counters.

• Ignoring duplicate model instances during predictor debug.