Part 1 answers: How do I make it fail on demand, and make the failure small?
A bug you cannot reproduce is a rumour. This part makes failures deterministic, records everything needed to replay them, and shrinks a failing test or design until the bug has nowhere to hide.
Replay any failure from a seed, a commit and a command line.
Shrink a failing random test to a handful of transactions automatically.
Make a flaky failure fail reliably.
8 lessons, about 86 min
The part on one page, taken from each lesson's Remember box.
Determinism and Seeds
A run is decided by its seed, test and plusargs, commit and local edits, simulator and version, and options. Change any one and you may get a different run.
Prove determinism: run the same command twice and compare the logs, not only the last line.
Give each stream of stimulus its own generator, owned by one process. A shared generator makes the numbers depend on process order.
Rule of thumb: a seed is a reproduction only together with the commit, the command line and the simulator that ran it.
Recording Everything to Replay
Record at the moment of failure, by script: commit hash, local edits as a patch, exact command, simulator version, environment, and the FAIL line.
Replay in a fresh checkout of the recorded commit, never in your working tree, and compare the FAIL line, not just pass or fail.
A replay that gives a different result means the record is missing an input, or the replay is broken. Test it once on a fresh record.
Rule of thumb: if a failure was not recorded when it happened, assume it cannot be replayed.
Shrinking a Failing Test
Write the signature first: what counts as the same failure. Too loose and you shrink into another bug; too strict and the result keeps noise.
Cut length first (bisect the operation count), then content (delta debugging on a recorded list). Each try must still be a legal test.
Delta debugging gives a list where no single operation can go, not the smallest possible list. Read it, then write the directed test.
Rule of thumb: shrink before you debug: a failure that takes one operation has very few places to hide.
Reducing the Design
Keep the suspect RTL files unchanged and compile them from where they live. A pasted, edited copy can lose the bug or grow a new one.
A stub keeps the interface rules (handshake, timing, one request in flight) and nothing else. Check the reduced bench fails with the same signature.
If the reduced bench passes, the cause is in what you cut: an interaction, not one block. Put blocks back one at a time.
Rule of thumb: a reduction counts only when it fails the same way as the full design.
Making a Flaky Test Fail
A different seed is a different test. A test is flaky only if the same seed, commit and command sometimes pass.
Measure the rate over hundreds of seeds and count by signature: 100 seeds can say 5% or 16% for the same 10% bug.
Raise the rate by biasing the stressor the mechanism needs. Running longer or wider rarely helps.
Rule of thumb: never call a test flaky until you have a failure rate, a signature and a guess at what the failing seeds share.
When Simulation Cannot Reproduce It
RTL simulation has no setup and hold, no analogue, no metastability and no environment. A bug that needs any of them will not appear there, whatever the seed.
Use the lab as an instrument: vary one condition at a time and watch the rate. What the rate follows points at the mechanism.
A model earns trust when the symptom appears with it on, disappears with it off, survives the fix, and predicts a new lab result.
Rule of thumb: if simulation cannot reproduce it, add the missing physics to the simulation as a model; never just add seeds.
Timeouts and Watchdogs in Benches
A global timeout caps a whole test; a watchdog catches a test that stopped making progress. Every bench needs both, counted in cycles.
Progress means useful work finished, not activity. A bench that polls a status register forever looks busy to a watchdog that counts any bus response.
Size limits from measurements: the longest legal test and the longest legal quiet stretch, with margin. Never raise a limit to make a failure go away.
Rule of thumb: a hang must end in a FAIL line that says what was waiting, within a few times the longest legal quiet stretch.
The Reproduction Checklist
Reproduce is finished when one command, at one commit, with one tool, fails the same way every time, quickly, and ends with the same FAIL line.
Record everything that changes the run: commit, command, seed, plusargs, simulator, options and environment.
Check the signature, not just FAIL. A smaller or faster repro that fails for another reason is a different bug.
Rule of thumb: do not start Observe until the repro command is written in the bug report and has failed twice in a row.
Practice question: Coverage-driven stimulus and targeted seeding
Practice question: Blocking vs non-blocking in sequential logic
Code lab: Seed plusarg
Code lab: Stochastic random stability
Practice question: Design a simple DMA controller
Interview: run a mock interview on this topic