DCRouter has been verified as one design: four ports, fixed FIFOs, CRC on, low power available. Now it is to be sold as IP. Customers will choose 2 to 8 ports, FIFO depths from 2 to 64, flit widths from 8 to 64 bits, and whether to include CRC and low power. Every lesson so far verified one point in that space. Each customer will tape out a different point, and many will be points nobody has simulated.
How do you verify a design that is really thousands of designs?
One obvious answer is to verify the default configuration thoroughly; another is to verify every configuration. The first ships bugs that only appear at the extremes or in combinations (lesson 1.6): a FIFO of depth 2 whose almost-full logic underflows, an arbiter that only fails with an odd number of ports. The second is impossible: even with each parameter reduced to a few representative values, the product grows too fast to build and regress.
With five parameters at four representative values each, every combination is 4^5 = 1,024 builds, each with its own regression. Every pair of values can be covered in far fewer: at least 16, because one pair of parameters alone has 16 value pairs. For four parameters with four values, 16 configurations cover all 96 pairs against 256 for every combination; for five parameters with three values, 11 suffice against 243 (fact M5). The number needed grows only slowly as parameters are added, which is what makes pairwise selection practical.
Pairs are not everything: NIST's studies found most real failures triggered by one or two interacting parameters, but some by up to six (fact M5). Pairwise is the starting point, raised to three-way coverage for the parameters that interact most.
Selection | Why |
|---|---|
Legal ranges defined and checked at elaboration | Illegal configurations rejected by the IP itself, so they are never shipped or verified |
Extremes: minimum and maximum of every parameter | Off-by-one and boundary bugs live at the edges (depth 2, 8 ports, width 8) |
Customer configurations first | The configurations that will become silicon get full regression |
Pairwise (or three-way) covering array | Every interaction of two settings exercised with few builds |
Random configurations in every nightly regression | Over months, many more points visited |
Formal on small instances | Proofs at small parameter values, where state spaces are tractable; structure that does not depend on size gives confidence beyond them |
The testbench is parameterised too: stimulus, reference model, checkers and coverage all read the configuration; a testbench hard-coded for four ports silently tests only four.
Coverage is per configuration and across configurations: which configurations were run, and which features were covered in each.
Features depend on configuration: CRC checks only when CRC is present; the plan states which evidence applies to which configurations.
Every customer configuration is signed off: by full regression on it, or by an argument from neighbouring configurations stated as residual risk (lesson 14.4).
Practice | Purpose |
|---|---|
Configuration generator producing covering arrays plus extremes | A deliberate, recorded set of builds |
Customer configurations added to regression on delivery | Silicon-bound points get full evidence |
Random configurations nightly, failures reproduced by configuration and seed | Broad exploration over time |
Parameter checks in the RTL | Illegal configurations cannot be built |
An IP team (illustrative) regressed its default configuration and the two largest customer configurations. A new customer chose a FIFO depth of 3, legal but not a power of two, which nobody had built. The pointer wrap logic assumed a power of two and overwrote an entry once every few thousand packets. A pairwise set including the odd depths, or a parameter check rejecting them, would have caught it before delivery.
A testbench hard-coded for one configuration silently exercises and checks only that configuration, whatever the design's parameters say.
At least 16, because one pair of parameters alone has 4 × 4 = 16 value pairs, each needing a configuration.
Configurable IP is a space of designs; neither the default nor every configuration works.
Regress extremes and customer configurations; cover pairs with covering arrays; explore randomly.
Parameterise the testbench too, and state which configurations the evidence covers.