Sign-Off Metrics & Review

No single number signs off a chip

Functional coverage proves planned scenarios happened. It cannot prove the plan was complete, that checks were watching, or that the design is stabilizing. Sign-off therefore reads four instruments together — each one covering the others' blind spots.

THE FOUR SIGN-OFF INSTRUMENTS

  instrument            proves                     blind spot it covers
  ────────────────────────────────────────────────────────────────────
  functional coverage   planned scenarios          — (this is the plan's
  (merged, vs plan)     occurred                   own view)
  code coverage         RTL structure executed     scenarios the PLAN
  (line/branch/toggle/  (incl. logic no plan       MISSED — unhit code
   FSM)                 row mentions)              means unplanned logic
  assertion status      checks were ACTIVE         passing by silence —
  (pass counts +        and fired on real          an assertion never
   cover properties)    traffic                    triggered checked nothing
  bug curve             discovery rate             high coverage with a
  (bugs/week trend)     is decaying                rising bug curve says
                                                   the plan is too small

  Sign-off = all four healthy, reviewed together, per feature.

How the instruments cross-check

• Functional 100% + a code-coverage hole = logic exists that no plan row exercises — plan gap, investigate before waiving.

• Code 100% + functional 60% = lines toggled but scenarios untried — code coverage's classic false comfort.

• Assertion with zero activations on merged runs = a check that never watched anything; its 'pass' is vacuous.

• Flat-lining bug curve + open coverage = stop arguing about the curve; the remaining holes are exactly where untested bugs live.

Per-feature beats aggregate — the 99% trap

Aggregate percentages average across the model, and averages hide zeros. Sign-off must read coverage per plan feature , because the aggregate weights bins by count, not by importance.

WHY 99% AGGREGATE CAN HIDE A 0% FEATURE

  feature            bins   hit    feature %
  ─────────────────────────────────────────
  burst shapes        420   420      100%
  address map         310   310      100%
  qos arbitration     250   250      100%
  error recovery       20     0        0%   ◄── the one that
  ─────────────────────────────────────────      escapes to silicon
  aggregate          1000   980       98%

  98% sounds done. Error recovery — 20 bins out of 1000 —
  contributes only 2 points to the aggregate while being
  the feature most likely to be hit in the field and least
  likely to be hit by random stimulus.

  Sign-off rule: report MIN(feature %), not just the aggregate.
  A feature at 0% is a veto regardless of the average.

This is also where option.weight earns its keep: weighting plan-critical groups higher makes the aggregate harder to game — but the per-feature table, not the weighted average, is what the review reads.

The final review

Sign-off review checklist

1. Merged functional coverage meets goal per feature — minimum feature percentage on the table, not just the aggregate.

2. Every zero-hit bin is closed, or carries a waiver with approver, reason, and spec link.

3. Code coverage (line/branch/FSM/toggle) meets target; unhit code is explained or traced to a new plan row.

4. Every assertion activated at least once in the merged regression; cover properties for key sequences hit.

5. Bug discovery curve flat for the agreed window (commonly 2+ weeks of full regression).

6. Exclusion-file diff since last review walked item by item — late additions get the most scrutiny.

7. Plan-to-spec delta reviewed: spec changes since the plan was written have corresponding plan rows.

8. Regression health: pass rate, no masked failures, seeds and test list archived with the merged database.

Interview angle — "when is verification done?": the honest senior answer is that 'done' is an engineering judgement supported by converging evidence, not a number: the plan traces to the spec, merged functional coverage meets per-feature goals with every hole closed or waived, code coverage corroborates with no unexplained unhit logic, assertions demonstrably fired, and the bug curve has flat-lined under sustained regression. The junior trap is answering '100% coverage' — which only proves the scenarios you thought of.

Key takeaways

• Sign-off reads four instruments — functional coverage, code coverage, assertion activity, bug curve — each covering the others' blind spots.

• Aggregate percentages hide zeros; the review reads per-feature numbers and treats a 0% feature as a veto.

• An assertion that never activated checked nothing — activation counts are part of coverage evidence.

• 'Done' is converging evidence plus judgement: plan traced, holes dispositioned, curves flat, exclusions defended.

Common pitfalls

• Signing off on the aggregate while a small-bin-count feature sits at 0%.

• Treating 100% code coverage as scenario evidence — it proves execution, not intent.

• Counting assertion passes without checking activation counts — vacuous passes look identical to real ones.

• Skipping the exclusion-diff walk at the final review — deadline exclusions hide exactly there.