Q&A: Methodology & Debug

Q: Directed vs constrained-random — when is each the right call?

Direct answer: directed tests encode scenarios you already know matter — reset sequences, boot flows, spec corner cases named in the plan — with full determinism and cheap debug. Constrained-random explores the space you did NOT think to enumerate, trading per-run debuggability for breadth; its real product is bug classes nobody predicted , at the cost of needing coverage to know what actually ran. Production methodology is the hybrid: constrained-random regression as the engine, directed tests for must-hit scenarios, and coverage holes converting back into new directed tests or tightened constraints.

Legend: [QA]

  DIRECTED vs CONSTRAINED-RANDOM                      [QA]

                     Directed            Constrained-random
  Scenario source    engineer's list     solver explores legal space
  Debug              trivial (det.)      needs seed + logging discipline
  Coverage/effort    linear              superlinear once env exists
  Finds              anticipated bugs    unanticipated interactions
  Blind spot         imagination limit   needs coverage to see itself

  CLOSURE LOOP: random regression → merge coverage →
                holes → directed test or constraint tweak → repeat

Follow-up you should expect

“Your random regression passes for a week — what does that tell you?” — Almost nothing without coverage: passing only says nothing sampled failed. The senior frame: constrained-random without functional coverage is unmeasured spray; the pass signal is meaningful only against a coverage model that says the space was actually visited.

Junior vs senior answer

• Junior: “random finds more bugs, directed is for simple stuff.”

• Senior: identifies what each uniquely buys (determinism vs unanticipated interactions), names the blind spots of both, and describes the closure loop that joins them.

Q: How do you reproduce a random failure?

Direct answer: re-run with the same seed, same code revision, same simulator version, same command line — random stability guarantees the identical value stream under exactly those conditions. The discipline that makes this work is set up before failures happen: every regression run logs its seed; failure triage records seed + revision + test name as the repro key; and nobody edits code between repro runs, because inserting one object construction or one $urandom call upstream re-seeds everything after it.

# Regression: per-run random seed, ALWAYS logged
simv +UVM_TESTNAME=bus_stress +ntb_random_seed=20260612 -l run_20260612.log

# Failure → exact repro with debug visibility added OUTSIDE the seed path
simv +UVM_TESTNAME=bus_stress +ntb_random_seed=20260612 \
     +UVM_VERBOSITY=UVM_HIGH -ucli -do waves.do

Follow-up you should expect

“The seed reproduces it, but the failure is 2 hours in — now what?” — Shrink the path to the bug: log the solved transaction stream and replay just the suspect window as a directed sequence; or bisect by constraining the random space toward the failing scenario (tighten constraints around the observed values) so it fires early. Converting a deep random failure into a short directed repro is the actual skill being probed.

Junior vs senior answer

• Junior: “rerun with the same seed.” — necessary, not sufficient.

• Senior: full repro key (seed+revision+version+command), the upstream-edit re-seeding hazard, and the shrink-to-directed technique for deep failures.

Q: Where do constraints belong — transaction, sequence, or test?

Direct answer: by what each layer knows . The transaction class encodes protocol legality — what can physically appear on the interface — as hard constraints, plus soft defaults for typical values. Sequences encode scenario shape — bursts, ordering, mixes — usually as inline with constraints on the items they generate. Tests encode intent — which sequences, which knobs, which overrides — through configuration and factory choices, mostly without writing raw constraints at all.

// TXN: legality (hard) + defaults (soft)
class bus_txn;
  rand bit [31:0] addr;
  rand bit [3:0]  len;
  constraint c_legal { len inside {[1:16]}; addr[1:0] == 2'b00; }
  constraint c_dflt  { soft len == 4; }
endclass

// SEQUENCE: scenario shape via inline constraints
class burst_seq;
  task body();
    repeat (20)
      `uvm_do_with(req, { len inside {[8:16]};
                           addr inside {[32'h8000:32'h8FFF]}; })
  endtask
endclass

// TEST: intent via knobs/factory — no raw constraints
//   uvm_config_db#(int)::set(this, "*", "region_sel", RAM_HI);
//   bus_txn::type_id::set_type_override(err_txn::get_type());

Follow-up you should expect

“What goes wrong when this layering is violated?” — Scenario constraints hardcoded into the transaction over-constrain every other test (the over-constrained-base anti-pattern); legality re-stated per-sequence drifts out of sync with the spec and lets illegal stimulus through whichever sequence forgot a clause. The boundary test: “would EVERY user of this class agree with this constraint?” — only then does it belong in the transaction.

Junior vs senior answer

• Junior: “put constraints in the transaction class.” — one layer, no rationale.

• Senior: maps legality/shape/intent onto txn/sequence/test, gives the every-user boundary test, and names the failure mode of each misplacement.

Q: How do you inject errors at a controlled rate?

Direct answer: make the error decision itself a rand field weighted by dist, with the rate as a state-variable knob the test sets — then let implications shape the corrupted fields when the decision fires. Two layers: the decision (weighted bit) and the corruption (constraints conditioned on it). Keeping the knob in config rather than hardcoded weights lets one env run clean smoke tests and hostile soak tests unchanged.

class err_txn;
  rand bit       inject_err;
  rand bit [7:0] crc;
  bit [7:0]      good_crc;          // computed reference, set pre-solve
  int unsigned   err_pct = 5;       // KNOB: test sets via config

  constraint c_rate { inject_err dist { 1 :/ err_pct,
                                        0 :/ (100 - err_pct) }; }
  constraint c_crc  { inject_err  -> crc != good_crc;     // corrupt
                      !inject_err -> crc == good_crc; }   // clean
endclass
// Smoke test: t.err_pct = 0;   Soak test: t.err_pct = 25;

Follow-up you should expect

“How does the scoreboard cope with injected errors?” — The transaction carries its own inject_err flag, so the monitor-side checker flips expectation: corrupted transactions must be rejected/flagged by the DUT, and an accepted corrupt transaction is the bug. Error injection without expectation switching just makes the scoreboard fail on your own stimulus — mentioning the checker side unprompted is the senior tell.

Junior vs senior answer

• Junior: “randomize a bit and corrupt the packet when it’s set.” — mechanism only, fixed rate, no checking story.

• Senior: knob-driven dist rate, implication-shaped corruption both arms covered, and the scoreboard expectation flip that makes the errors checkable.

Q: How do you know your randomization is actually covering the space?

Direct answer: you don’t — until you measure it with functional coverage . Constraints define what is possible ; coverage measures what occurred . The two are linked by the solver’s distribution: legal-but-improbable corners (skewed implications, tiny dist tails, narrow sub-ranges inside wide ones) can take astronomically many seeds to hit by chance. Covergroups sampled on the monitor stream, merged across the regression, are the only honest answer to “did we exercise it?”

// The constraint says jumbo is POSSIBLE:
constraint c_size { size dist { [64:128] :/ 95, [8960:9216] :/ 5 }; }

// Coverage says whether jumbo OCCURRED — and which jumbo:
covergroup cg with function sample(pkt_txn t);
  cp_size : coverpoint t.size {
    bins small = {[64:128]};
    bins jumbo_lo = {[8960:9000]};
    bins jumbo_hi = {[9201:9216]};   // narrow tail — watch this one lag
  }
endgroup

Follow-up you should expect

“Coverage shows a hole after 500 seeds — what are your options, in order?” — First check reachability (is the hole even legal under current constraints — a contradiction makes it permanently unhittable); then bias with dist weights or a dedicated sequence toward the hole; then write a directed test if the corner is singular; finally, if analysis proves it unreachable, exclude it with documentation. Constraint debugging and coverage closure are one loop, not two activities — that sentence is the senior summary interviewers remember.

Junior vs senior answer

• Junior: “add a covergroup and check the percentage.”

• Senior: separates possible from occurred, predicts WHICH bins lag from the constraint distribution, and walks the hole-triage ladder from reachability check to documented exclusion.

Key takeaways

• Constrained-random + coverage + directed-for-the-holes is one closure loop, not three techniques.

• Repro key = seed + revision + simulator version + command line; log it before you need it.

• Layer constraints by knowledge: txn legality, sequence shape, test intent.

• Error injection = knob-weighted decision bit + implication-shaped corruption + scoreboard expectation flip.

• Constraints define possible; coverage measures occurred — never conflate them.

Common pitfalls

• Random regressions with unlogged seeds — every failure becomes unreproducible folklore.

• Scenario policy hardcoded in transaction classes — every future test fights the base class.

• Error injection without checker awareness — the scoreboard flags your own stimulus as DUT bugs.

• Calling a passing-but-unmeasured random regression “verified” — passing only covers what ran.