Self-Checking Tests & Regression

Overview

A testbench that requires a human to stare at waveforms is not a verification environment — it is a debug aid. Self-checking means every test decides its own PASS/FAIL verdict from scoreboard and checker results, prints it in a machine-greppable banner, and returns an exit code automation can act on. Once tests are self-checking, you can run hundreds of them in parallel across seeds — that is a regression, and it is how real projects find bugs at scale.

This topic covers the discipline end to end: the PASS/FAIL contract, seeds and reproducibility, runtime knobs via $value$plusargs, a leveled logging strategy that triage scripts can parse, and the regression flow that turns one testbench into a nightly bug-finding machine.

Sub-topics

1. Self-Checking Discipline — PASS/FAIL banners, error counting, max-error abort, exit codes.

2. Seeds & Reproducibility — per-run seeds, logging the seed, replaying failures, seed sweeps.

3. Test Knobs via Plusargs — $test$plusargs / $value$plusargs, verbosity, txn-count, error-rate knobs.

4. Logging & Verbosity Strategy — leveled logger class, scoped prefixes, grep-able conventions.

5. The Regression Flow — test-list x seeds matrix, dispatch, aggregation, triage, rerun loop.

FROM ONE TEST TO A REGRESSION

  single test                       regression
  ───────────                       ──────────
  testbench                         test list x seed matrix
     │                                  │
     ▼                                  ▼
  scoreboard counts errors          dispatch N sims in parallel
     │                                  │
     ▼                                  ▼
  final banner:                     scrape banners + exit codes
  "TEST PASSED" / "TEST FAILED"         │
     │                                  ▼
     ▼                              aggregate: pass / fail / error-sig
  $finish with exit code                │
                                        ▼
                                   triage buckets → rerun failures
                                   (same seed = same failure)

Key takeaways

• Every test must end in an unambiguous PASS/FAIL banner driven by its own checks.

• A failure is only useful if the seed is logged and the run is replayable.

• Knobs + seeds + logging conventions are what make one TB scale to a thousand runs.