Skip to content
DenseChip
Learn
PracticeCode labInterview

DenseChip

Navigate

Menu

LearnPracticeCode labInterview

Tools

All toolsSV editorCode repoResume
DenseChip

Practice, courses, and chip-design tools in one focused workspace.

LearnPracticeCode labInterviewToolsLearning paths

© 2026 DenseChip

AboutContactPrivacyTerms
Skip to lesson

Course contents

  • Part 0 · What Debugging Is6
  • Part 1 · Reproduce8
  • Part 2 · Observe9
  • Part 3 · Localise10
    • The First Wrong Cycle
    • Bisection in Time
    • Bisection in Space
    • Bisection in Stimulus
    • Bisection in History
    • Bisection Across Configurations and Seeds
    • Cone-of-Influence Tracing
    • Differential Debugging
    • When the Bug Is in the Test
    • The Localisation Checklist
  • Part 4 · Hypothesise and Test6
  • Part 5 · Bug Classes16
  • Part 6 · Debug by Stage9
  • Part 7 · Hangs and Deadlocks7
  • Part 8 · Intermittent Failures7
  • Part 9 · Fix, Prove, Prevent7
  • Part 10 · Case Studies12
  • Part 11 · Debug as a Team5
  • Part 12 · Interview Prep5
Loading lesson

‹Part 3 · Localise
  1. Topics
  2. The Art of Hardware Debug
  3. Part 3 · Localise
  4. Bisection in Space

Part 3 · Localise · Working (W)

Bisection in Space

Halve the design instead of the run: interface monitors at block boundaries along the failing path, read by the time of their first breach. A real dbglab failure at the stream pins narrowed to one line of the power domain in six runs.

On this page

    Why this matters

    A failure seen at a chip's outputs could come from any block it passed through, and reading them one by one costs days. Monitors at block boundaries split the path in half per run, and the earliest breach points at the block that made the first wrong value.

    After this lesson you can: list the blocks on a failing path, place interface monitors that check each boundary's own rules, read their first-breach times, and find the block whose inputs are clean while its outputs are wrong.

    Note

    Level: Working (W) · Debug-loop step: Localise

    Before this: The First Wrong Cycle, Assertions as Always-On Probes

    Unknown at the pins, after a power cycle

    dbglab's periph_power test switches the stream peripheral's power domain off with words waiting, then back on. It fails at the stream pins:

    Terminal
    $ tb/run.sh periph_power 1
    dbglab: test=periph_power seed=1
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown

    A stream word passes through the bus fabric, the peripheral's registers, an async FIFO into a second clock domain, a synchroniser and the switchable power domain with its sink and isolation clamps: nine RTL modules between the CPU and the pins. Any of them could have made the first unknown value. Which one?

    What people usually do, and why it fails

    The usual move is to start at the pins and read backwards through the RTL, module by module, or to start with the block that seems most likely ("it is a power test, so it is the power gate") and read that. Reading backwards is slow, and the most likely block is a guess that may be right for the wrong reason. Walking back cycle by cycle from the pins works, as in the first wrong cycle, but each hop needs you to understand the block you are stepping through, and nine modules across two clock domains is a long walk.

    The design's hierarchy is an ordered path for this failure, just as time is an ordered axis. You can halve it.

    The method: monitors at boundaries, halve the path

    1. List the blocks on the failing path in data-flow order, from where the data enters to where the symptom is seen.

    2. Put a monitor on the boundary in the middle. A monitor checks only that interface's own rules: controls are never unknown, data is known whenever it is valid, words leave in the order they arrived. It needs no model of the rest of the design.

    3. Record the first breach and its time, not just pass or fail. Run each monitor on the same failing run.

    4. Blame the side that drives the breaching signal. A FIFO's read-enable is an input of the FIFO but an output of its reader. The driver is the suspect, whichever way the data flows.

    5. Compare times. The earliest breach anywhere on the path wins. A later breach elsewhere may be an echo: wrong values travel backwards through feedback such as pointers, ready signals and credits.

    6. Stop at the block whose inputs were clean when its outputs first breached, then halve inside it the same way, one level down the hierarchy.

    Each monitor run splits the suspects, so a path of N blocks takes about log2(N) runs, plus one or two to confirm. On the debug loop this is still Localise: you are finding where, not yet why.

    The picture: four blocks, three monitors, one echo

    The peripheral's path with the first breach at each boundary. The earliest, at cycle 74, is on the FIFO's read port, on a signal that periph_pd drives; the breach at cycle 90 on the write port is an echo carried back by the read pointer.

    Read it from the earliest number. M3 breaches first, at cycle 74, and the signal it caught comes out of periph_pd. M2 breaches later, at 90, on a signal computed from the FIFO's full flag, which depends on the read pointer: that is the same unknown value coming back across the FIFO.

    Do it: five monitors and one look inside

    The monitors

    One module holds all five monitors; +MON turns one on. Each samples its interface mid-cycle, when every input is stable, and prints its first breach and nothing else. It only reads the design.

    if_mon.sv
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    // not-run: needs dbglab's sources and bench; compile it next to them (command in the Evidence box)
    // if_mon: interface monitors at the block boundaries on the peripheral's path. +MON=<name> turns
    // one on; it reports the first breach of its interface's own rule and nothing else. Reads only.
    //   slave   periph_slave's bus side: req, write, addr known; wdata known on a write
    //   fifo_wr async FIFO write port: winc known; wdata known when winc
    //   fifo_rd async FIFO read port: rinc, rempty known; rdata known when a word is popped
    //   pd_in   periph_pd's inputs: pwr_on, iso_en, en, rempty known
    //   pd_out  periph_pd's outputs: rinc, out_valid known; out_data known when out_valid
    `timescale 1ns/1ps
    module if_mon;
      string mon;
      bit    hit = 1'b0;
      initial if (!$value$plusargs("MON=%s", mon)) mon = "";
    
      task automatic breach(input string what);
        if (!hit) $display("MON %s: first breach at cycle %0d (%0.1f ns): %s", mon, dbglab_tb.cycle, $realtime, what);
        hit = 1'b1;
      endtask
    
      // bus-clock interfaces, sampled mid-cycle when every input is stable
      always @(negedge dbglab_tb.clk) if (dbglab_tb.rst_n) begin
        if (mon == "slave") begin
          if ($isunknown(dbglab_tb.dut.u_per.req)) breach("req unknown");
          else if (dbglab_tb.dut.u_per.req && ($isunknown(dbglab_tb.dut.u_per.write) || $isunknown(dbglab_tb.dut.u_per.addr)))
            breach("write or addr unknown");
          else if (dbglab_tb.dut.u_per.req && dbglab_tb.dut.u_per.write && $isunknown(dbglab_tb.dut.u_per.wdata))
            breach($sformatf("wdata = %h", dbglab_tb.dut.u_per.wdata));
        end
        if (mon == "fifo_wr") begin
          if ($isunknown(dbglab_tb.dut.u_per.u_fifo.winc)) breach("winc unknown");
          else if (dbglab_tb.dut.u_per.u_fifo.winc && $isunknown(dbglab_tb.dut.u_per.u_fifo.wdata))
            breach($sformatf("wdata = %h", dbglab_tb.dut.u_per.u_fifo.wdata));
        end
      end
      // peripheral-clock interfaces
      always @(negedge dbglab_tb.pclk) if (dbglab_tb.prst_n) begin
        if (mon == "fifo_rd") begin
          if ($isunknown(dbglab_tb.dut.u_per.u_fifo.rinc) || $isunknown(dbglab_tb.dut.u_per.u_fifo.rempty))
            breach($sformatf("rinc = %b, rempty = %b", dbglab_tb.dut.u_per.u_fifo.rinc, dbglab_tb.dut.u_per.u_fifo.rempty));
          else if (dbglab_tb.dut.u_per.u_fifo.rinc && !dbglab_tb.dut.u_per.u_fifo.rempty
                   && $isunknown(dbglab_tb.dut.u_per.u_fifo.rdata))
            breach($sformatf("rdata = %h", dbglab_tb.dut.u_per.u_fifo.rdata));
        end
        if (mon == "pd_in") begin
          if ($isunknown(dbglab_tb.dut.u_per.u_pd.pwr_on) || $isunknown(dbglab_tb.dut.u_per.u_pd.iso_en)
              || $isunknown(dbglab_tb.dut.u_per.u_pd.en) || $isunknown(dbglab_tb.dut.u_per.u_pd.rempty))
            breach($sformatf("pwr_on = %b, iso_en = %b, en = %b, rempty = %b", dbglab_tb.dut.u_per.u_pd.pwr_on,
                   dbglab_tb.dut.u_per.u_pd.iso_en, dbglab_tb.dut.u_per.u_pd.en, dbglab_tb.dut.u_per.u_pd.rempty));
        end
        if (mon == "pd_out") begin
          if ($isunknown(dbglab_tb.dut.u_per.u_pd.rinc) || $isunknown(dbglab_tb.dut.u_per.u_pd.out_valid))
            breach($sformatf("rinc = %b, out_valid = %b (pwr_on = %b, iso_en = %b)",
                   dbglab_tb.dut.u_per.u_pd.rinc, dbglab_tb.dut.u_per.u_pd.out_valid,
                   dbglab_tb.dut.u_per.u_pd.pwr_on, dbglab_tb.dut.u_per.u_pd.iso_en));
          else if (dbglab_tb.dut.u_per.u_pd.out_valid && $isunknown(dbglab_tb.dut.u_per.u_pd.out_data))
            breach($sformatf("out_data = %h", dbglab_tb.dut.u_per.u_pd.out_data));
        end
      end
      final if (!hit && mon != "") $display("MON %s: clean through cycle %0d", mon, dbglab_tb.cycle);
    endmodule

    Cut 1: the FIFO, the middle of the path

    The async FIFO sits between the bus side and the peripheral's own clock domain, so its two ports split the path in half. Run both monitors on the failing run:

    Terminal
    $ vvp -n build/mon.vvp +TEST=periph_power +SEED=1 +MON=fifo_wr | grep -E '^(MON|FAIL)'
    MON fifo_wr: first breach at cycle 90 (900.0 ns): winc unknown
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown
    $ vvp -n build/mon.vvp +TEST=periph_power +SEED=1 +MON=fifo_rd | grep -E '^(MON|FAIL)'
    MON fifo_rd: first breach at cycle 74 (740.0 ns): rinc = x, rempty = 1
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown

    Both ports breach. Taken as yes or no, that says nothing. Taken by time, it says a lot. The read port breaks first, at cycle 74 (740 ns), on rinc, the read-enable, while the FIFO is still empty. rinc is an input of the FIFO, but it is driven by periph_pd. The write port breaks 16 cycles later on winc, which periph_slave computes from the FIFO's full flag, and the full flag depends on the read pointer that rinc moves. So the write-side breach is the read-side value coming back.

    (The pclk monitors sample mid-cycle, on the falling edge, so each reports a change at the next pclk sample. rinc actually goes unknown at 725 ns, bus cycle 73, when the test powers the domain off; that is the number the cone-of-influence lesson reads from the dump, and the same lag turns rempty's 87 into the 89 below. A probe that samples on the pclk rising edge, as in the debug log, first sees it at 759 ns, cycle 76. Only the order matters here, and it is the same in every view.)

    Cut 2: confirm the bus side is clean

    Terminal
    $ vvp -n build/mon.vvp +TEST=periph_power +SEED=1 +MON=slave | grep -E '^(MON|FAIL)'
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown
    MON slave: clean through cycle 431

    Nothing unknown ever reaches the peripheral from the bus. The fabric, the CPU port and everything upstream are cleared for this failure, and so are periph_slave's registers as a source: their inputs are clean and their first bad output (winc, at 90) comes after the FIFO's read port went bad.

    Cut 3: periph_pd's own boundary

    Terminal
    $ vvp -n build/mon.vvp +TEST=periph_power +SEED=1 +MON=pd_in | grep -E '^(MON|FAIL)'
    MON pd_in: first breach at cycle 89 (888.0 ns): pwr_on = 0, iso_en = 1, en = 0, rempty = x
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown
    $ vvp -n build/mon.vvp +TEST=periph_power +SEED=1 +MON=pd_out | grep -E '^(MON|FAIL)'
    MON pd_out: first breach at cycle 74 (740.0 ns): rinc = x, out_valid = 0 (pwr_on = 0, iso_en = 1)
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown

    Read the two times together. periph_pd's output rinc goes unknown at cycle 74. Its inputs are clean at 74; the first unknown input, rempty, arrives at 89, and it is the FIFO reacting to the bad rinc 15 cycles earlier. The domain is powered off (pwr_on = 0) and isolated (iso_en = 1), exactly as the test intends, so its control inputs are right. Clean inputs, wrong output: periph_pd is the block. Its other output, out_valid, is clamped to 0 as it should be.

    One level down: inside periph_pd

    periph_pd holds the sink, a model of the supply switch, and the isolation clamps. A probe prints its internal nets at each peripheral clock around the breach:

    Terminal
    $ ../probes/run-probe.sh ../probes/pd_inside.sv periph_power 1
    dbglab: test=periph_power seed=1
    cycle 67: pwr_on=1 iso_en=0 | rinc_s=0 valid_s=0 | rinc_p=0 valid_p=0 | rinc=0 out_valid=0
    cycle 70: pwr_on=1 iso_en=1 | rinc_s=0 valid_s=0 | rinc_p=0 valid_p=0 | rinc=0 out_valid=0
    cycle 74: pwr_on=0 iso_en=1 | rinc_s=0 valid_s=0 | rinc_p=x valid_p=x | rinc=x out_valid=0
    ...
    FAIL test=periph_power seed=1 cycle=431: STREAM: pstream_valid is unknown

    The three columns are the three layers. The sink's own outputs (_s) are 0: it is held in reset while unpowered. After the supply model (_p), both are unknown at cycle 74, which is what an unpowered domain's outputs are meant to be. After the clamps, out_valid is 0 and rinc is still unknown. One output is isolated; the other is not:

    design.sv
    1
    2
    3
    4
    5
    // snippet: rtl/periph_pd.sv at HEAD, lines 33-36
      // Isolation: every output of the domain is clamped to 0 while iso_en is high.
      assign rinc      = rinc_p;          // internal: only feeds our own async FIFO
      assign out_valid = iso_en ? 1'b0  : valid_p;
      assign out_data  = iso_en ? 32'h0 : data_p;

    The comment says rinc is internal, but it leaves the switched domain: the FIFO's read side is always powered.

    Run

    Boundary

    First breach

    Driven by

    Conclusion

    1

    FIFO write port

    cycle 90, winc

    periph_slave, from the FIFO's full flag

    later than run 2: an echo

    2

    FIFO read port

    cycle 74, rinc

    periph_pd

    earliest: suspect periph_pd

    3

    periph_slave bus side

    none

    the fabric

    bus side cleared

    4

    periph_pd inputs

    cycle 89, rempty

    the FIFO

    inputs clean at 74

    5

    periph_pd outputs

    cycle 74, rinc

    periph_pd

    the block

    6

    inside periph_pd

    cycle 74, after the clamps

    line 34

    rinc is not clamped

    What the evidence says

    Six runs took a failure at the pins at cycle 431 to one line of one module at cycle 74, without reading the fabric, the registers or the FIFO. The bench's check fired 357 cycles after the first wrong value, in a different clock domain and on a different signal (pstream_valid, after power-up), which is why reading back from the pins was going to be slow. The next step is Hypothesise: an unclamped `rinc` from an unpowered domain moves the FIFO's read pointer; it predicts that the read pointer becomes unknown as soon as words are waiting, and that clamping rinc makes the test pass. Testing and fixing it belong to power-intent bugs and the missing isolation case study.

    Traps

    Common pitfall

    Following the data flow instead of the driver. The read port is downstream of the FIFO, so it is tempting to say "the problem is after the FIFO". But rinc flows the other way: from the reader into the FIFO. Ask who drives the breaching signal, not where the boundary sits on the data path.

    Common pitfall

    Reading monitors as yes or no. Both FIFO ports breach, and periph_pd's inputs breach too. Without times, three of five monitors fail and every block looks guilty. Wrong values echo backwards through pointers, ready signals and credits. Only the earliest breach, with clean inputs at that moment, names the block.

    Common pitfall

    Trusting a monitor you have not checked. The first version of the bus-side monitor tested two signals with one $isunknown on their concatenation, and in Icarus it reported a breach at cycle 8 that a plain print of the same signals at the same moment did not show. Before you act on a monitor's first breach, print the signals it names at that cycle.

    Why this matters

    Senior lens. Space bisection is cheapest when the monitors already exist. Interface checkers written once per interface type (valid/ready, FIFO ports, a power-domain boundary) and bound everywhere turn this hunt into reading a log: in a bench that ran all five monitors all the time, the first failing run would have printed the cycle-74 breach on its own.

    Power-domain boundaries deserve a standing check: no unknown value on any signal that leaves a switched domain while it is isolated. That one rule, written as a monitor or an assertion, catches every missing clamp, including the ones a design comment calls "internal".

    Interview

    Asked: "A test fails at the chip's outputs and the data passes through eight blocks to get there. How do you find the block that is wrong without reading all eight?"

    Weak answer: I would start from the output and trace backwards through the RTL of each block until I find something that looks wrong, starting with the block I think is most likely.

    Strong answer: I halve the path. I list the blocks in data-flow order and put an interface monitor on the boundary in the middle, checking only that interface's rules, like no unknown controls and valid data when valid. I record each monitor's first breach and its time. The earliest breach wins, and I blame the side that drives the breaching signal, because wrong values can echo backwards through feedback like FIFO pointers or ready. The block whose inputs were clean when its outputs first breached is the one to open, and I halve inside it the same way. That is three or four runs for eight blocks.

    Follow-ups: (1) Two monitors breach at the same cycle. How do you decide? (2) What if the interface has no written rules?

    Check yourself Had you run only the FIFO write-port monitor and seen its breach at cycle 90 on winc, where might you have gone next, and what would have saved you?

    winc is computed by periph_slave, so you might have opened periph_slave or blamed the bus side. Two checks save you: winc is made from the FIFO's full flag, so ask what drove that; and run the monitor on the other side of the FIFO, whose earlier breach (cycle 74) shows the value came from the read side. A monitor result is a time and a driver, not a direction.

    Check yourself Suppose periph_pd's input monitor had breached at cycle 70, before its outputs went bad at 74. What would that change?

    Then periph_pd might only be passing on a wrong input, and the block to suspect is whatever drove that input at cycle 70. You would read the breach message to see which input it was and walk to its driver (the register bits from periph_slave, or the FIFO's rempty). A block is guilty only when its inputs were clean at the moment its outputs first went wrong.

    Remember

    • A monitor checks its interface's own rules (known controls, known data when valid, order), so it needs no model of the whole design.

    • Compare first-breach times, not yes or no. A wrong value can echo backwards through feedback, like FIFO pointers, and breach upstream later.

    • The block to open is the one whose inputs were clean at the moment its outputs first went wrong. Then halve inside it the same way.

    Rule of thumb: blame the side that drives the breaching signal at the earliest breach, never simply the upstream side.

    How this was checked

    dbglab at HEAD 49c5f04, Icarus Verilog 13.0, 2026-10-17. tb/run.sh periph_power 1 fails at cycle 431 (STREAM: pstream_valid is unknown). if_mon.sv compiled next to dbglab (iverilog -g2012 -Wno-timescale -I tb -o build/mon.vvp <rtl/files.f> tb/bus_rules.sv tb/dbglab_tb.sv if_mon.sv) and run five times with +MON=slave|fifo_wr|fifo_rd|pd_in|pd_out gave the lines shown; pd_inside.sv printed periph_pd's internal nets.

    Monitor bug found on the way: a first version tested write and addr with one $isunknown on their concatenation and reported a breach at cycle 8, while a print of the same signals at the same moment showed them known (write=0 addr=00003004). One $isunknown per signal reported clean. All monitors shown use one call per signal.

    Limits: one bug, one test, Icarus only. In Verilator the supply model drives noise instead of X, so X monitors stay silent and the same test hangs instead (see bisection across configurations). Monitors find where a wrong value appears, not why.

    Keep going

    • Practice: Isolation cells and boundary wrapper: clamp logic and X containment

    • Practice: Isolation cell modelling

    • Code lab: Isolation cell model

    • Code lab: Passive monitor only

    • Interview: run a mock interview on this topic

    Related topics

    • Part 3 · Localise

    • Next: bisection in stimulus

    • Bisection in time

    • Reducing the design: cutting the hierarchy for speed

    • CDC and RDC: debug and case studies

    Previous

    26 / 107

    Next: Bisection in Stimulus

    Up next

    Up nextBisection in Stimulus

    Sign in to save progress

    Practice this lessonReport an error

    See also

    • The First Wrong Cycle
    • Bisection in History
    • Bisection Across Configurations and Seeds