A verification lead turns on full waveform dumping for the nightly "so the waves are there when we need them". The next morning the shared disk is full, half the regression died writing files, and the runs that finished took three times longer than usual. The one failure anyone looked at needed 2 microseconds of six signals. How do you get exactly the waves a failure needs, and nothing else?

The idea: dump on the rerun, not in the regression

A waveform dump records the value of every selected signal at every change, so you can see what the design did over time. It is the most useful debug artifact and the most expensive one: dump size grows with the number of signals times the number of changes, and writing it slows the simulation. The rule at scale is simple. Regression runs dump nothing. When triage picks a run to debug, you replay it from its bundle (the same seed reproduces it exactly) and dump only that rerun, only the part of the design you need, and only the time window around the failure.

A security camera system works like this. Nobody stores high-resolution footage of every corridor forever. Cameras keep low-cost logs, and when an incident is reported, someone pulls the footage for one corridor around the time of the incident. The log tells you where and when to look; the footage shows what happened.

The picture

What to dump, and when

Build it: scoped, windowed dumping on the DMA bench

The standard system tasks

IEEE 1800 defines the VCD tasks: $dumpfile("name") opens the file, $dumpvars(levels, scope) selects signals (levels = 0 means everything below the scope, 1 means that module only), and $dumpoff / $dumpon pause and resume recording. Each vendor has its own compressed format with its own tasks or commands: FSDB for Synopsys Verdi ($fsdbDumpfile, $fsdbDumpvars), WLF for Questa, SHM for Xcelium, and the older VPD for VCS. The ideas (scope, depth, time window) are the same in all of them.

The scratch top below is the bench's own top module with one addition: a plusarg that turns dumping on, starting at a given time. Without the plusarg, nothing is dumped, so the same executable serves the regression and the debug rerun.

1
2
3
4
5
6
7
8
9
10

// snippet: added to a copy of the bench's top module (scratch file)
  // +DUMP_FROM=<ps>: start dumping the DUT at that time (0 = from the start).
  // Without the plusarg nothing is dumped, so regression runs pay nothing.
  longint dump_from;
  initial
    if ($value$plusargs("DUMP_FROM=%d", dump_from)) begin
      #(dump_from * 1ps);
      $dumpfile("dump.vcd");
      $dumpvars(0, top.dut);
    end

Triage pointed at dma_rand_test seed 16 on the mismatch build, first error at 345,000 ps. The rerun starts dumping 100 ns earlier.

Watch it run (scratch runs, Verilator 5.048 with --trace)

$ ./Vtop +UVM_TESTNAME=dma_rand_test +verilator+seed+16
  (no dump file)
$ ./Vtop +UVM_TESTNAME=dma_rand_test +verilator+seed+16 +DUMP_FROM=0
  dump.vcd: 26360 bytes, 334 timestamps (5000 .. 1665000 ps), 140 signals
$ ./Vtop +UVM_TESTNAME=dma_rand_test +verilator+seed+16 +DUMP_FROM=245000
  dump.vcd: 23372 bytes, 285 timestamps (250000 .. 1665000 ps), 140 signals
  UVM_ERROR dma_tb.sv(396) @ 345000: uvm_test_top.env.sb [SB_MISMATCH] expected 4735a210, got b8ca5def

  same bench built with a trace config that keeps only top.dut:
$ ./Vtop ... +DUMP_FROM=0        dump.vcd: 18519 bytes, 111 signals
$ ./Vtop ... +DUMP_FROM=245000   dump.vcd: 16602 bytes, 111 signals

Three things to read in those numbers, and one surprise:

• The window works. Starting at 245 ns drops everything before it, and the replayed run hits the same mismatch at 345,000 ps, so the waves show the exact failure triage found.

• The DMA is tiny, so the savings are small: 140 signals over 1.7 microseconds is about 26 KB of VCD. Scale the same arithmetic to a subsystem with 500,000 signals and a 10-millisecond test, and a full VCD dump is in the range of tens to hundreds of gigabytes. Compressed formats (FSDB, WLF, SHM) are typically several times to an order of magnitude smaller than VCD, which helps but does not change the rule.

• The surprise: this Verilator build ignored the scope argument and `$dumpoff`. $dumpvars(0, top.dut) still recorded all 140 signals of the design, and a $dumpoff / $dumpon pair in a separate scratch test did not pause recording. A delayed $dumpvars call did start the dump late, which is why the window above is built that way. On Verilator, scope is chosen at build time instead: --trace-depth, or a configuration file with tracing_off -scope "top" and tracing_on -scope "top.dut", which produced the 111-signal dumps (the remainder are package-level variables). Commercial simulators honour the scope and $dumpoff; check yours with a two-line test before you rely on either.

Dumping on the first error

Some teams make the rerun automatic: when triage marks a run for debug, the farm reruns it with +DUMP_FROM set to the first-error time minus a margin. A related trick turns dumping on at the first error from inside the bench: a report catcher (see report catchers) that, on the first UVM_ERROR, calls $dumpvars and lets the run continue for a while. That records what happens after the error, which is what you need for a hang or a cascade, but not what led up to it; for a mismatch, the replay with a window that starts before the error is the better tool.

What breaks: the dump that changed the bug

A team found that a failing run passed when they reran it with waves on. It was not flaky: the bundle was identical except for one plusarg that enabled dumping, and enabling it changed the order in which two processes ran at the same simulation time. The bench had a race (a monitor sampled a signal in the same time step a driver changed it, without a clocking block), and the dump's extra work shifted the scheduling. Two lessons follow. A waveform switch is part of the bundle, so record it. And a failure that disappears when you observe it points at a race: look for signals read and written in the same time step, which is the problem the bench's clocking-block rule (war story 1) exists to prevent.

Next: the bugs are being found and fixed. Metrics and Sign-off is how you show the project is converging.

Related topics

• Up: Regression and Sign-off

• Hang Playbook

• Transaction recording