Reading a merged report

Hole analysis starts from the merged regression database, never a single seed. The report lists every bin with its merged hit count, and a hole is a bin below its at_least threshold.

Read holes grouped by covergroup and cross, because holes cluster. Ten empty cells sharing one source bin usually mean one cause, not ten.

• UNREACHABLE: the scenario is impossible. Action: illegal_bins when the spec forbids it, otherwise ignore_bins plus a documented waiver, not a test.

• NEEDS-NEW-TEST: reachable, but no stimulus aims there. Action: a directed test or a new sequence.

• NEEDS-CONSTRAINT-CHANGE: existing random stimulus reaches it once its constraints or distribution change. Action: retune constraints, no new test.

• WRONG-BIN: the model is buggy, through a bad range, wrong signal, or wrong sample event. Action: fix the model, since past data for that bin is void.

The triage flow

Order matters: checking the bin (Q1) costs minutes, and a directed test (Q4) costs days. Never pay Q4 prices for Q1 problems.

Q2 is the step that involves people. Reachability questions are answered fastest by the designer, who knows whether the combination shares logic or is structurally cut off.

Worked triage of five holes

• Five holes, four different classes. Treating every hole as “write more tests” would waste effort on four of the five.

• Hole 4 is the dangerous one: the model was wrong, so the past coverage of that area was fiction.

• Hole 5 is not empty. Below-at_least bins are holes too, and the fix is distribution, not existence.

• Hole 3 is a spec-forbidden cell left in the auto product. A spec NO becomes illegal_bins, never ignore_bins; writing it into every cross at plan time would have prevented the triage cost.

Testbench

The bench counts the error coverpoint from the report above in three seeds with the bins model, at_least 32, then merges them the way a regression merge does: the same bins, hit counts summed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// uses: covmodel
// Three seeds merged: a hole in one seed is often hit in another, and a bin below at_least is still a hole.
module tb_hole_merge;
  cm_point seed[3], merged;
  function automatic cm_point make(string n);
    cm_point p = new(n);
    p.add("parity", 0, 0); p.add("timeout", 1, 1); p.add("crc", 2, 2); p.add("overflow", 3, 3);
    p.at_least = 32;
    return p;
  endfunction
  initial begin
    // Hits per seed for parity, timeout, crc and overflow.
    int hits[3][4] = '{'{80, 0, 2, 0}, '{70, 0, 9, 40}, '{61, 0, 7, 0}};
    foreach (seed[s]) begin
      seed[s] = make($sformatf("cp_err seed %0d", s + 1));
      foreach (hits[s][b]) repeat (hits[s][b]) seed[s].sample(b);
    end
    // The merge: the same bins, hit counts summed over every seed.
    merged = make("cp_err merged");
    foreach (seed[s]) foreach (merged.bin_list[b]) merged.bin_list[b].hits += seed[s].bin_list[b].hits;
    seed[0].report(); merged.report();
    if (seed[0].n_covered() != 1 || merged.n_covered() != 2 || merged.bin_list[2].hits != 18) $fatal(1, "FAIL: merge math");
    $display("PASS: seed 1 alone showed 3 holes; merged, overflow closed, crc sat at 18 of 32 and timeout at 0");
    $finish;
  end
endmodule

  coverpoint cp_err seed 1: 25.00% (1 of 4 bins)  below goal 100
    parity         80
    timeout        0     <- hole
    crc            2     <- hole
    overflow       0     <- hole
  coverpoint cp_err merged: 50.00% (2 of 4 bins)  below goal 100
    parity         211
    timeout        0     <- hole
    crc            18    <- hole
    overflow       40
PASS: seed 1 alone showed 3 holes; merged, overflow closed, crc sat at 18 of 32 and timeout at 0

Related topics

• Cross coverage

• Covergroup options and methods