A reviewer asks for one more cross in DCRouter's coverage model: source input × destination × priority × every length × fault type × FIFO occupancy. Each coverpoint is modest. The cross has 4 × 4 × 2 × 16 × 3 × 9 = 13,824 bins. At the coupon collector's rate (lesson 4.2) that needs well over a hundred thousand random packets if every bin were equally likely, and they are not: high occupancy and faults are rare, so many bins would never close at all.
Which crosses should a coverage model contain, and how do you keep them closable?
The obvious instinct is to cross everything that might interact, because bugs live in combinations (lesson 1.6). But the number of combinations multiplies, and so does the effort to fill them, while most of them exercise logic that does not interact at all. A model that cannot close produces a coverage number nobody believes and holes nobody reads.
Cross for DCRouter | Bins | Comment |
|---|---|---|
dst × prio × length class (4 classes) | 4 × 2 × 4 = 32 | Lesson 6.9's header cross: closes in tens of tests |
dst × prio × every length | 4 × 2 × 16 = 128 | Lesson 4.2: about 695 uniform packets |
src × dst × prio × length class | 4 × 4 × 2 × 4 = 128 | Adds the source; does any rule depend on it? |
src × dst × prio × every length × fault × occupancy | 13,824 | Unclosable, and mostly combinations of logic that does not interact |
Technique | How it shrinks the cross | DCRouter example |
|---|---|---|
Group values into classes | Values that behave alike share one bin | Lengths 2 to 14 as one class: 16 values become 4 |
Cross only what interacts | Combine coverpoints that share logic or a rule | Priority × destination matters (arbitration); source × CRC value does not |
Ignore or mark illegal | Remove combinations that cannot or must not occur | Bad CRC on a length-0 packet |
Cross with a condition | Sample the cross only in the situation that matters | Occupancy × push/pop only when a push arrives |
Pairwise coverage | Every pair of values, not every tuple (fact M5) | Four 4-valued configuration parameters: 16 configurations cover all pairs, against 256 |
The question for every cross is the same: which logic would behave differently in this combination than in its parts? If none, the cross measures nothing new.
dst × prio × length class: yes, arbitration (R6) and length handling (R3 to R5) meet at the header.
fault type × priority × length class: yes, error reporting differs by priority (BUG_13) and length (BUG_12).
prio_en × contention: yes, as events: priority only matters when packets compete (lesson 5.10).
source input × CRC value: no, no logic depends on the pair.
occupancy × every length: no; occupancy × push/pop event, sampled on a push, instead.
Practice | Purpose |
|---|---|
Cross reviews asking which logic each cross exercises | Crosses that measure interaction, not volume |
Grouping, ignore and illegal bins as a matter of style | Models sized to what a regression can close |
Pairwise or covering arrays for configuration spaces | All pairs covered with few configurations |
A team (illustrative) added a 6-way cross of transaction attributes to their model and saw overall coverage drop from 95% to 41%. For the rest of the project the number was quoted with an asterisk, real holes in small crosses were lost among thousands of irrelevant empty bins, and the cross was eventually deleted without anyone examining it. Two well-chosen 2-way crosses would have measured the interactions that mattered and closed.
4 × 4 × 2 × 4 = 128; only if some rule or risk depends on the source together with the rest. DCRouter's per-flow ordering does involve the source, so a source × destination cross can be justified; crossing it with priority and length adds little.
That the design treats all middle lengths alike; if a length-dependent bug lived at, say, length 8, grouping would hide it, so the assumption must come from the specification and the design.
A cross has as many bins as the product of its coverpoints, and closing it costs about n ln n samples or far more.
Cross only what shares logic; group values, ignore impossible combinations, condition on events.
Use pairwise coverage for configuration spaces.