No single number signs off a chip

Functional coverage proves planned scenarios happened. It cannot prove the plan was complete, that checks were watching, or that the design is stabilizing.

Sign-off therefore reads four instruments together, each covering the others' blind spots.

Sign-off is all four healthy, reviewed together, per feature. The code-coverage blind spot runs the other way too: Building Your First Coverage Model runs a FIFO test with Verilator line coverage, where every line and both sides of every branch ran while the corner bin that mattered read 0.

• Functional 100% plus a code-coverage hole means logic exists that no plan row exercises. That is a plan gap, so investigate before waiving.

• Code 100% plus functional 60% means lines toggled but scenarios untried. That is code coverage's classic false comfort.

• An assertion with zero activations on merged runs never watched anything. Its pass is vacuous.

• A flat bug curve plus open coverage ends the argument about the curve. The remaining holes are exactly where untested bugs live.

Per-feature beats aggregate

Aggregate percentages average across the model, and averages hide zeros. Sign-off must read coverage per plan feature, because no aggregate weights features by importance.

98% sounds done, and 75% reads like ordinary slow progress. Neither number names the missing feature. Error recovery, 20 bins out of 1000, moves the bin-count total only 2 points, yet it is the feature most likely to be hit in the field and least likely to be hit by random stimulus.

The sign-off rule: report the minimum feature percentage, not just the aggregate. A feature at 0% is a veto regardless of the average.

This is also where option.weight earns its keep: weighting plan-critical groups higher makes the aggregate harder to game. The per-feature table, not the weighted average, is still what the review reads.

The final review

1. Merged functional coverage meets goal per feature: minimum feature percentage on the table, not just the aggregate.

2. Every zero-hit bin is closed, or carries a waiver with approver, reason, and spec link.

3. Code coverage meets target, and unhit code is explained or traced to a new plan row.

4. Every assertion activated at least once in the merged regression, with cover properties hit for key sequences.

5. Bug discovery curve flat for the agreed window, commonly two weeks or more of full regression.

6. Exclusion-file diff since last review walked item by item. Late additions get the most scrutiny.

7. Plan-to-spec delta reviewed: spec changes since the plan was written have corresponding plan rows.

8. Regression health confirmed: pass rate, no masked failures, seeds and test list archived with the merged database.

Testbench

The bench builds the four features from the table with the bins model, hits every bin of the first three and none of error recovery, and reads the three numbers a sign-off report could show.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// uses: covmodel
// Four plan features: the bin-count total, the LRM rollup and the minimum feature, from the same hits.
module tb_signoff;
  cm_group dma = new("dma coverage model");
  cm_point feat[4];
  initial begin
    string names[4] = '{"burst shapes", "address map", "qos arbitration", "error recovery"};
    int    nbins[4] = '{420, 310, 250, 20};
    int    total = 0, hit = 0;
    real   lowest = 100.0;
    foreach (feat[f]) begin
      feat[f] = new(names[f]);
      feat[f].add_each("b", 0, nbins[f] - 1);
      if (f != 3) for (int v = 0; v < nbins[f]; v++) feat[f].sample(v);   // error recovery never ran
      dma.points.push_back(feat[f]);
      total += feat[f].n_bins(); hit += feat[f].n_covered();
      if (feat[f].get_coverage() < lowest) lowest = feat[f].get_coverage();
      $display("  feature %-16s %4d bins %4d hit  %6.2f%%", names[f], feat[f].n_bins(), feat[f].n_covered(), feat[f].get_coverage());
    end
    $display("  bin-count total   %0d of %0d = %0.2f%%", hit, total, 100.0 * hit / total);
    $display("  LRM rollup        %0.2f%%", dma.get_coverage());
    $display("  minimum feature   %0.2f%%%s", lowest, lowest == 0.0 ? "  <- veto" : "");
    if (100.0 * hit / total != 98.0 || dma.get_coverage() != 75.0 || lowest != 0.0) $fatal(1, "FAIL: sign-off math");
    $display("PASS: 98%% by bin count and 75%% by rollup, and only the minimum names the 0%% feature");
    $finish;
  end
endmodule

  feature burst shapes     420  bins 420  hit  100.00%
  feature address map      310  bins 310  hit  100.00%
  feature qos arbitration  250  bins 250  hit  100.00%
  feature error recovery   20   bins 0    hit    0.00%
  bin-count total   980 of 1000 = 98.00%
  LRM rollup        75.00%
  minimum feature   0.00%  <- veto
PASS: 98% by bin count and 75% by rollup, and only the minimum names the 0% feature

Related topics

• Covergroup basics

• Coverage bins