Part 11 answers: How do I lead a hunt and make the team faster?
On a real project, bugs cross owners: RTL, verification, physical design and the lab. This part covers triage that converges, hand-offs that do not lose information, metrics that help, and designing for debuggability before tape-out.
Run a triage meeting that ends with owners and next experiments.
Choose bug-tracking fields and metrics that help rather than hurt.
Review a design for debuggability before tape-out.
5 lessons, about 58 min
The part on one page, taken from each lesson's Remember box.
Triage Meetings That Converge
Prepare before the meeting, with scripts: signatures, one repro each, the first bad commit and the files it touched. The meeting is for decisions.
One row per signature, not per failing run. 66 red lines here are five bugs; too coarse or too fine a grouping both mislead.
Every row leaves with one named owner, one experiment, its prediction and a due time. A signature without them is the meeting's failure.
Rule of thumb: nobody debugs in the triage meeting; the meeting assigns the next experiment and moves on.
Ownership and Hand-Offs
A bug has one owner at a time: the person who acts next. Ownership moves when the receiver accepts, not when the ticket is reassigned.
Hand over the hunt, not just the failure: replay, signature in every simulator, first wrong cycle with the probe that shows it, what is ruled out, the question.
The hand-off ends when the receiver reproduces the first wrong cycle on their own machine. Until then the giver still owns it.
Rule of thumb: never reassign a bug the receiver has not yet reproduced.
Bug Tracking and Severity
Severity is the harm if it ships; frequency is how often a test sees it; priority is the order of work. Three fields, set by different questions.
Judge severity by silicon, not by the simulator: an unreset register that one simulator hides is still random data on a real chip.
Fixed is not closed. Verified means the repro and a wider sweep pass at the fix, run by someone or something other than the author's local tree.
Rule of thumb: set severity by asking what the bug does in silicon in the worst reachable case, never by how many seeds fail.
Debug Metrics That Help, and Ones That Hurt
Measure intervals in a bug's life, not counts of bugs. Report the median and the 90th percentile; the tail is where the expensive bugs live.
Pair every speed metric with a quality one: closed and stayed closed, verified at the fix, found at the earliest stage that could find it.
A metric used to rank people stops measuring the work. Use debug metrics to find slow steps in the process, never as individual targets.
Rule of thumb: never set a target on a number that the people being measured can move without finding a bug.
Designing for Debuggability
Write the debug spec from the failure modes: for each, the register the lab reads and the rule that turns the value into a decision.
A diagnosis needs independent witnesses. A counter that the bug corrupts cannot tell you the bug is there; two counts kept apart can.
Debug logic is verified like function: inject each failure, check the decoded registers name it, and mutate the debug logic to find untested promises.
Rule of thumb: every failure mode in the review gets a register, a decision rule and a test that shows the rule picks the right culprit.
Practice question: Debugging rare seed failures in regression
Practice question: Report catcher triage: noise down, real failures up
Practice question: Seed reproducibility: failing to log random seeds breaks debug
Code lab: Watchdog FSM
Code lab: Outstanding tracker
Interview: run a mock interview on this topic