Skip to content
DenseChip
Learn
PracticeCode labInterview

DenseChip

Navigate

Menu

LearnPracticeCode labInterview

Tools

All toolsSV editorCode repoResume
DenseChip

Practice, courses, and chip-design tools in one focused workspace.

LearnPracticeCode labInterviewToolsLearning paths

© 2026 DenseChip

AboutContactPrivacyTerms
Skip to lesson

Course contents

  • Start Here6
  • SystemVerilog Language Foundations28
  • Object-Oriented SystemVerilog for Verification30
  • Constraint Randomization — Interview-Critical Deep Dive36
  • SystemVerilog Assertions — Interview-Critical Deep Dive40
  • Functional Coverage in SystemVerilog22
  • Testbench Architecture in SystemVerilog23
  • Advanced Language & Integration25
  • Senior & Interview Preparation21
  • Principal-Level Verification28
    • Reusable Testbench Architecture
    • Checking Out-of-Order Traffic
    • Assertion Depth
    • Coverage Modelling for Real Features
    • Capstone: Out-of-Order Memory Controller
    • Principal Judgment
      • Escaped Bug Analysis
      • Intermittent Failures
      • Metrics That Lie
      • Scaling Verification
      • System Design Interviews
Loading lesson

‹Principal Judgment
  1. Topics
  2. SystemVerilog
  3. Principal-Level Verification
  4. Principal Judgment
  5. Escaped Bug Analysis

Part 9 · Principal-Level Verification · Advanced

Escaped Bug Analysis

Trace an escaped bug back to the plan, stimulus, coverage and checking gaps, then close the class of bug.

On this page

    An escape is a bug that verification should have found and did not. The fix in the RTL is the easy part. The valuable work is finding which part of the verification flow failed, so the same class of bug cannot get through again.

    The illustrative case

    This is a made-up scenario. A chip link uses credit-based flow control: the sender starts with 4 credits, spends one per packet, and the receiver returns one when it frees a buffer. In the lab, the link runs at full rate for hours, then throughput drops and the link stops. A reset brings it back.

    Lab debug reads the credit counter over the debug bus: it shows 0 while the receiver reports all buffers free. Credits are leaking. The RTL had this:

    design.sv
    1
    2
    3
    4
    5
    // snippet: the escaped counter update
    always_ff @(posedge clk or negedge rst_n)
      if (!rst_n)       credits <= 4'(MAX);
      else if (consume) credits <= credits - 1;
      else if (ret)     credits <= credits + 1;   // lost when consume is also high

    When a packet leaves in the same cycle a credit comes back, the else if drops the return. Each overlap leaks one credit. Four overlaps and the link is dead.

    Ask four questions, in order

    Each question names one layer of the flow. The first layer that answers "no" is where the escape happened, but check all four: escapes usually pass through more than one gap.

    Question

    In this case

    Gap

    Was the feature in the plan, with the right closure criterion?

    The plan row said "credits reach 0 and MAX". It did not mention same-cycle spend and return.

    Plan

    Could the stimulus create the condition?

    The receiver model returned credits only in idle cycles, to keep the model simple. Spend and return never met.

    Stimulus

    Did coverage measure the condition?

    Coverpoints on the credit value and on each signal alone. No cross of consume and ret.

    Coverage

    Would a checker have seen the symptom?

    Tests were short, so one leaked credit never stalled anything. No end-of-test audit that all credits came home.

    Check

    All four failed. That is typical. A single gap is usually covered by another layer; an escape needs the gaps to line up.

    Prove each gap with a bench

    The bench below is the regression that closes the escape. It does three things the original did not: it lets spend and return overlap, it counts how often they overlap and fails if they never do, and it drains the link at the end and audits the credit count.

    credit_ctr.sv
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    module credit_ctr #(parameter int MAX = 4) (
      input  logic       clk,
      input  logic       rst_n,
      input  logic       consume,   // a packet leaves: spend one credit
      input  logic       ret,       // the far side frees a buffer: credit back
      output logic [3:0] credits,
      output logic       can_send
    );
      always_ff @(posedge clk or negedge rst_n)
        if (!rst_n) credits <= 4'(MAX);
        else        credits <= credits - 4'(consume) + 4'(ret);   // both in one cycle: net zero
      assign can_send = (credits != 0);
    endmodule
    

    The OLD_STIM switch rebuilds the original bench inside the new one, so each gap can be shown on its own. These results come from running the bench with Verilator 5.048:

    Change

    Result

    What it shows

    None (fixed DUT, new stimulus)

    PASS, 95 overlaps in 400 cycles

    The condition is exercised and checked.

    Put the else if counter back

    FAIL at cycle 6, the first overlap

    The model compare catches the bug as soon as the stimulus reaches it.

    Buggy DUT and OLD_STIM = 1

    Model compare and drain audit pass; only "consume and return never met" fires

    The original stimulus could not reach the bug. Without the coverage check, this run would be green.

    Fixed DUT and OLD_STIM = 1

    FAIL on the same coverage line

    The hole is reported even when the RTL is correct. That is the point: a hole is a defect in the bench.

    The third row is the escape, reproduced: a green regression on buggy RTL. The coverage check is written as a pass condition, not as a report someone has to read.

    Fix the class, not the instance

    The RTL fix is one line. The useful output of an escape review is a list of actions that each close a gap for every similar design:

    1. Plan: every counter with independent increment and decrement gets a plan row for the same-cycle case.

    2. Stimulus: reference models of the far side may not serialise events the real design can overlap. Review every model shortcut for this.

    3. Coverage: cross the inputs of every up/down counter. The cross is cheap and the hole is visible.

    4. Check: every resource that is borrowed and returned (credits, tags, buffers, IDs) gets an end-of-test audit after a drain.

    5. Sweep: search the RTL for else if chains that update one register from two independent events, and review each.

    The sweep matters most. If one engineer wrote this pattern once, it probably exists elsewhere in the same codebase.

    What not to do

    • Do not stop at "add a directed test for the overlap". That test protects one counter. The audit and the cross protect all of them.

    • Do not blame the designer. The verification flow is meant to catch design mistakes; the review is about the flow.

    • Do not accept "it needed hours of lab time to show up" as a reason. Hours of runtime were needed to exhaust the credits, but the first leak happened within microseconds. An audit sees the first leak.

    Common pitfall

    Writing the escape report around the symptom (the hang) instead of the first wrong event (the first lost credit). The symptom tells you how long the bug took to hurt. The first wrong event tells you which check should have fired.

    Interview

    Question: "A bug escaped to silicon on a block you verified. What do you do?" Answer: reproduce it in simulation first; then walk plan, stimulus, coverage and checks to find every gap it passed through; fix each gap for the whole class of bug; sweep the design for the same pattern; and show the new regression failing on the old RTL.

    Check yourself The new overlap test fails on the old RTL. Is that enough to close the escape review?

    No. It proves one directed case. The review closes when the plan, stimulus models, coverage and audits are fixed for every borrowed-and-returned resource, and the pattern sweep is done.

    Check yourself Why does the bench fail when OLD_STIM = 1 even with correct RTL?

    Because the overlap count is a pass condition. A stimulus that cannot create a required condition is a bench defect, whether or not the RTL happens to be correct.

    Related topics

    • Principal Judgment (topic)

    • Next: Intermittent Failures

    • Hole Analysis and Triage

    • Spec to Verification Plan

    Previous

    255 / 259

    Next: Intermittent Failures

    Up next

    Up nextIntermittent Failures

    Sign in to save progress

    Practice this lessonReport an error

    See also

    • Metrics That Lie
    • Scaling Verification
    • System Design Interviews