The three case studies were read from outside, through public facts. A team's own escapes can be read from inside, with every log, waveform and decision available. That is the post-mortem, and it is where most of a verification organisation's learning either happens or is lost.
How do you write a post-mortem that actually changes what happens next time?
The obvious post-mortem records the bug, the fix and a new regression test. That stops this bug from returning and nothing else. Post-mortems also fail in two other common ways: by naming a person ("the engineer did not run the test"), which teaches people to hide escapes, and by listing actions ("improve coverage") with no owner, no date and no way to know they were done.
Section | Content | Common failure |
|---|---|---|
Summary | What escaped, where it was found, impact | Too long, or missing the impact |
Timeline | When introduced, when it could have been caught, when found, when fixed | Skipped, so the gap between introduction and detection is never seen |
Mechanism | The bug's trigger and effect, with evidence (waveform, log) | Hypotheses written as facts |
Why not caught | For each technique: unreached, masked, unchecked or unspecified (lesson 11.7) | Stops at the first cause |
Similar risks | Where else the same class can occur (the sweep) | Missing: the next escape is the cousin |
Actions | Each with an owner, a date and a check that it is done | Vague, unowned, never closed |
Blameless: describe decisions and conditions, not people; the question is what made the escape possible.
Several whys, through the techniques: why did the stimulus not reach it, why did no check see it, why was it not in the plan?
Facts and hypotheses separated, as in lesson 16.1.
Suppose DCRouter had been signed off with the random-hunt testbenches of Parts 4 and 5 but without Part 6's coverage model, and a customer found that PKT_CNT3 never clears (BUG_16: the clear condition skips port 3). In the course's final bug-catch matrix (lesson 14.6), BUG_16 is caught only by the covered hunt, so this is a realistic escape for a team without that coverage.
Section | Content |
|---|---|
Summary | Writing PKT_CNT3 does not clear it; ports 0-2 are correct. Found by a customer's driver; counter values for port 3 are wrong after clearing. No data corruption. |
Timeline | Introduced in the counter rewrite; reachable by any test from then on; found after delivery; fixed in the next release. |
Mechanism | The clear condition contains k != 3, a leftover from an earlier debug change; evidence: register write and read-back on port 3. |
Why not caught | Stimulus: random tests wrote counter registers rarely and never cleared every counter. Checking: no check compared each counter with the model after a clear. Coverage: no bin for "PKT_CNT3 cleared", so nobody saw the hole. Plan: "clear each counter" was a feature with no evidence. |
Similar risks | ERR_CNT clears; any per-port register written in a loop; any condition with a port-specific exception. |
Actions | Add a check of every counter after every clear (owner: checker lead, this sprint). Add per-port clear bins to coverage (coverage owner, this sprint). Review all per-port registers for port-specific conditions (designer, two weeks). Add "every per-port register exercised per port" to the plan template (verification lead, next project). |
Notice that the fix to the RTL is one line and is not an action: it is already done. Every action is about why it escaped and where else it can happen, and each has an owner and a way to check it is done. The covered hunt in lesson 6.9 contains exactly these checks and bins, which is why it catches BUG_16.
Practice | Purpose |
|---|---|
A post-mortem for every escape past the planned stage | Learning is routine, not reserved for disasters |
Blameless format agreed in advance | People report escapes early and fully |
Actions tracked like bugs, reviewed until closed | Changes actually happen |
Lessons added to plan templates and checklists | The next project starts with them |
A team (illustrative) wrote thorough post-mortems, each ending with a list of improvements. A year later, a review found that fewer than a third of the actions had been done, and two escapes in the year belonged to classes already analysed. The documents were good; the actions had no owners and nobody tracked them. Post-mortem actions that are not tracked like bugs are not actions.
It is already done and only stops this bug; the actions address why it escaped and where similar bugs could be.
An owner, a date and a check that shows it is done.
A post-mortem's value is its actions, not its description of the bug.
Analyse why it was not caught per technique, and sweep for similar risks.
Keep it blameless, give every action an owner and a date, and track them to closure.