Two weeks before tape-out the program manager stops you in the corridor: the dashboard says 97% coverage and the nightly is mostly green. Are we done? You know the honest answer is not a number. You also know that "it depends" will end the conversation badly. In a senior interview this question arrives as a scenario, and the interviewer is listening for whether you can turn a feeling about risk into evidence someone else can check.
This lesson works the closure and sign-off questions that senior loops ask most, each as the interviewer poses it, with a strong answer built on what the course bench actually measured. The mechanisms have their own lessons; this page is about answering under pressure.
Think of a building inspection. The inspector does not ask "did you build a lot?". They ask whether each item on the code was checked, by what method, and who signed it. Verification sign-off is the same argument with more witnesses: the plan (every feature has items), functional coverage (stimulus reached each item), checking (a failure there would have been caught), code coverage (no RTL went unexercised), trend (bugs stopped arriving) and the other methods (formal, gate-level, CDC, power). A strong answer names the witnesses and admits which one is weakest.
The most common senior mistake is treating coverage as proof of checking. Coverage says the stimulus went somewhere. It says nothing about whether anything would have complained if the design misbehaved there. The course bench has a measurement of exactly that gap, and it is the best closure story you can tell.
Do not answer in general. Classify real holes. Here is the course bench's checking test with its named plan tally (on Verilator the covergroup percentages read 0.0% because the tool ignores coverpoints, so the bench prints which plan items were hit; on a commercial simulator the covergroups give the same picture):
+UVM_TESTNAME=dma_check_test
UVM_INFO dma_tb.sv(1530) @ 1305000: uvm_test_top.env.cov [COV] plan items hit: 9 of 17
UVM_INFO dma_tb.sv(1531) @ 1305000: uvm_test_top.env.cov [COV] plan holes: len=0 len=16 len>16 irq_en=0 outcome=err clear=err gap=0 tlast=early
UVM_INFO dma_tb.sv(411) @ 1305000: uvm_test_top.env.sb [SB] 14 words matched, 0 mismatched
UVM_INFO dma_tb.sv(1639) @ 1305000: uvm_test_top.env [CMP] in-order comparator: 14 matches, 0 mismatchesA strong answer walks the holes and gives each its class:
Hole | Class | What closes it |
|---|---|---|
| Constraint never generates it | A directed test: |
| Constraint range stops at 16 | Widen the range; |
| Soft constraint never overridden | A test that overrides the soft constraint and polls instead |
| Needs error stimulus | The truncation test (early TLAST) produces ERR; sample it |
| The driver cannot produce it | Fix the stream driver: it always drops TVALID between beats |
| Needs error stimulus | Same truncation override |
The line that lifts the answer to senior is the gap=0 row. No amount of seeds closes it, because the stream driver drops TVALID after every beat, so back-to-back beats never happen. Coverage exposed an agent limitation, and the DUT's most common real-world traffic pattern was never tested. Say that, and say that you would fix the driver before writing a single closure test.
No, and the bench can show why. A fault campaign forced one stuck-at fault at a time inside the DMA and reran two tests. A fault counts as detected if the run ends with any error:
fault campaign (scripts/tools/uvm-bench/fault-campaign.sh)
fault dma_rand_test dma_truncate_test
1 detected(E11,F0) MISSED
2 detected(E8,F0) detected(E2,F0)
3 detected(E1,F0) MISSED
4 detected(E0,F1) detected(E0,F1)
5 MISSED detected(E0,F1)
6 MISSED MISSED
7 MISSED MISSED
8 MISSED MISSED
faults detected by at least one test: 5 of 8Three faults escaped every test, and the reasons are the strong answer:
Fault 6, DONE stuck at 1. Nothing checks that DONE rises only when the last word is accepted. The coverage model samples DONE; no checker owns its timing.
Fault 7, TLAST stuck at 0. The scoreboard compares data words and never looks at TLAST. Coverage of TLAST can be full while TLAST is never checked.
Fault 8, LEN bit 4 stuck at 0. Random lengths 1–16 never set bit 4 in these runs. That one is a stimulus hole the coverage model should have shown.
Fault 1 escaped one test by luck: the truncation test's random KEY had bit 0 = 0, so a stuck-at-0 on that bit changed nothing. Detection depends on stimulus, which is why seeds matter.
The answer to "are you done" is then: coverage says the stimulus reached the features; a fault campaign or a review of checker ownership says whether a bug there would be seen. You would add an assertion for DONE timing and a TLAST compare, rerun the campaign, and only then call the checking complete. Functional Safety: Fault Injection covers the campaign itself.
This is the question the review of this course found most often answered badly. The strong answer is a checklist with thresholds, each item owned and dated. The exact numbers are team policy; what the interviewer wants is that you have them.
Witness | Sign-off criterion (typical) | Why it is there |
|---|---|---|
Plan | 100% of plan items mapped to coverage or a test; every waiver reviewed, cited and approved | Nothing in the spec is unowned |
Functional coverage | Target met on the merged database across seeds (often 100% of P0 items, agreed % of the rest) | Stimulus reached the features |
Code coverage | Line, branch, toggle, FSM and expression targets met, with every exclusion reviewed by design | Catches RTL no plan item mentioned |
Checking | Every feature has a named checker; fault or mutation sampling shows they fire | Coverage does not prove checking |
Bugs | Zero open P1s; find rate flat for several weeks while stimulus still changes | The trend, not one green night |
Regression | Pass rate held across the last N nightlies on fresh seeds; no unexplained intermittents | Green on one seed proves one seed |
Other methods | Formal proofs closed (or bounded depth justified), GLS smoke passing, CDC, RDC, lint, X-prop and power checks clean | Simulation cannot sign these alone |
Then name the weakest witness out loud. "Our code coverage is excellent, our bug trend flattened last week, but our error-path checking is thin and I would want the fault campaign rerun first." That sentence is what the hiring manager remembers. Metrics and Sign-off is the home lesson for the metrics; Gate-Level Simulation and Formal Alongside Simulation cover the other methods.
The review of the old bank found this scenario promised and never walked. Here is a strong answer, in the first person, the way you would say it:
"First I stop the bleeding of attention: nobody debugs anything until the failures are bucketed. I group them by first-error signature (message ID plus the normalised text), which usually turns sixty red tests into three or four problems. For each bucket I check whether it is new: did it pass on the last good build with the same seeds? If yes, I bisect commits between the two builds on one seed, which takes hours, not days. I give each bucket one owner and a reproducer, and I separate testbench failures from design failures before anyone talks about schedule. Then I report three things to the program: how many distinct problems, which ones are design bugs, and my estimate of the risk each carries for tape-out. If a design bug lands this late I also ask what else that change touched, because late fixes are where escapes come from."
The follow-ups are predictable: how do you bucket automatically? (Regression Triage by Signature), what if it fails on one seed in fifty? (exact replay with the seed and build, Seeds and Exact Replay), and when do you escalate? (a design bug in a P0 feature, or a bucket you cannot reproduce).
A waiver is an audit record, not a favour. A strong answer separates a reviewed waiver from a silenced checker, and the bench has both. dma_waiver_test demotes one message ID matching one pattern, only on the scoreboard, tags each one [WAIVED DMA-142] and counts them, so the log still says five mismatches were waived under a ticket. dma_false_pass_test demotes SB_EMPTY for the whole scoreboard, unconditionally, and the verdict reads PASS for a test that compared nothing. Same API family, opposite discipline.
A waiver names the plan item or message, the reason with a spec reference, the approver, a date, and what alternative evidence covers the risk.
It is scoped to the smallest component and pattern that works, and it is counted in the log.
If the hole is real and reachable, propose a directed test or a tier change (weekly instead of nightly) before agreeing to a waiver.
Never "everything a bit". Rank by risk: features new in this design, features with past escapes, and features software depends on first stay. Breadth goes first: fewer configurations, fewer random seeds on stable features, coverage targets lowered on P2 items with the risk written down. Control-heavy blocks may move to formal, which closes faster on small state spaces. Then you say in writing what was not verified and what a bug there would cost. The interviewer is checking that you trade scope, not quality, and that the trade is visible.
A directed test. When you can name the preconditions, surgical stimulus is faster than another random sweep, and it documents the case. Keep the scoreboard and assertions on, so the closure also checks something.
No. Driver-side sampling records what the bench intended to send, not what the DUT accepted. A stalled or dropped beat still counts. Sample from monitors so coverage reflects the pins.
The weakest witness and its plan: for example, "Plan and coverage are complete, bugs have been flat for three weeks, but error-path checking is thin; one more fault campaign by Friday closes it." A number without the weak spot is not an answer.
Next: the behavioural round asks how you handled the people side of these same moments. Behavioural Round.