Two weeks before tape-out, a project meeting argues for an hour about whether verification is done. One engineer points to 100% functional coverage, another to three open bugs, a third to a bug curve that found two bugs last week. Nobody had agreed in advance what "done" would mean, so each fact counts for whatever its speaker wants. Criteria agreed at planning time end that argument before it starts.
What are your sign-off criteria?
The obvious single criterion, usually coverage or pass rate, is easy to measure and easy to satisfy without being done (lesson 9.5). Each metric has a blind spot, and only a set of criteria, each covering another's blind spot, gives a defensible answer.
Criterion | What it shows | Blind spot covered by |
|---|---|---|
Every planned feature has evidence | Nothing in the specification is unverified | Coverage and reviews (the plan may be incomplete) |
Functional coverage closed: holes hit, proved unreachable or waived | Planned situations were exercised | Checks (exercised is not checked) |
Code coverage reviewed, exclusions justified | No RTL left unexecuted without a reason | Functional coverage (values, combinations) |
Formal properties proved, or bounded results with justified depth | Key rules hold for all inputs | Simulation (rules not written as properties) |
Planted bugs or mutations caught | The checks actually detect bugs | Coverage (detection, not exercise) |
Bug discovery rate below a threshold for several weeks, with effort sustained | The design is maturing | Capture-recapture estimates and technique changes (Part 12) |
No open critical or major bug without a decision | Known problems are handled | Triage quality (lesson 12.4) |
Regression passing on the final RTL | Nothing broke at the end | Change management (lesson 9.7) |
Reviews complete: plan, coverage model, checkers, waivers | Independent eyes on the evidence | - |
Criterion | Status at the end of Part 13 |
|---|---|
Every feature F1–F28 has evidence | Three features open: F25, F26, F28 (closed in lesson 14.6) |
Coverage holes decided | 4 holes merged across 12 seeds: lp_en 0 (closed by F25), three ERR_CNT saturations (waiver, lesson 14.3) |
Formal properties proved | FIFO, arbiter and credit properties proved for every depth (lesson 7.9) |
All 24 Bug Zoo bugs caught | Yes, by the covered hunt (lesson 6.9) |
No open critical bugs | None open in the course's design |
Agree them at planning, with the people who will sign (lesson 9.1).
Make each one checkable: a number, a list, a state, not "adequate".
Include detection, not just exercise: planted bugs, mutations or proofs.
Allow decided exceptions: waivers with owners, not silent gaps (lesson 14.3).
Review them when the plan changes: new features need new criteria.
Practice | Purpose |
|---|---|
Sign-off checklists per block and per chip | The same questions asked every time |
Dashboards computing criteria from the plan and results | Status measured, not argued |
Criteria tightened from past escapes (lesson 11.7) | Each project learns from the last |
A team (illustrative) signed off on three criteria: coverage, pass rate and no open critical bugs. All three were met. None measured whether the checks could detect anything; a scoreboard for one interface had been disabled months earlier during debug. A planted-bug or mutation criterion would have shown that interface's checks catching nothing.
Planted-bug or mutation detection: bugs on that interface would go uncaught.
A rate can fall because effort fell (people moved, regression broken), not because the design matured.
Agree checkable sign-off criteria at planning, with the people who will sign.
Use a set that covers each other's blind spots, including detection, not just exercise.
Allow decided exceptions (waivers), never silent gaps.