The interviewer slides the DMA's one-page spec across the table. Walk me through your testplan. You have a good story: all seventeen coverage items in the bench's tally hit in the merged regression, every test green. Then they ask the follow-up you were hoping to avoid: the course's fault campaign broke the DMA eight different ways and the bench noticed five. Which rows of your plan should have caught the other three?
This lesson writes the DMA testplan from the spec: IDs, checks, coverage, tests and a status column filled from the course's real runs, gaps included. Then it shows how a reviewer reads a plan and how a plan stays tied to regression data.
A testplan (also called a verification plan) is the document that defines when verification is done. It breaks the spec into features, and each feature into plan items: single, checkable promises such as "a zero-length job finishes at once with DONE". For every item it records the check that proves the behaviour is correct, the coverage item that proves the behaviour actually happened in a passing run, the tests that create it, and its status, which comes from regression results rather than from the author's optimism.
Think of a building inspector's checklist. Each line cites a clause of the building code, names how it is verified (a measurement, a pressure test, a photo) and records the evidence. A line that says "plumbing: looked fine" fails review, because it has no clause and no evidence. A plan item with coverage but no check is that line: it proves the pipe exists, not that it holds water.
A useful plan is traceable (every spec sentence maps to an item with a stable ID that coverage, tests and tickets use) and honest (status is measured, and gaps are written down with a reason and an action).
Start from the spec and list features: the data transform, job lengths, error handling, status and interrupt, register rules, stream handshake and reset. Then turn every sentence of the spec, including the fine print, into an item. The IDs use a letter per feature, and the test names drop the dma_ prefix and _test suffix to fit. The check and coverage columns name the bench's real components: `dma_scoreboard`, the in-order comparator fed by dma_predictor, and the plan-item tally in `dma_coverage`.
ID | Plan item (the promise) | Check / coverage item | Tests | Status |
|---|---|---|---|---|
D1 | Each output word is the input word XOR KEY, in order | Scoreboard; comparator / len bins | smoke, rand, check | Pass |
D2 | TLAST on the LEN-th output word, and only there | None: data is compared, TLAST is not / none | None | Gap: fault 7 escaped |
D3 | LEN = 0 finishes at once: DONE, COUNT = 0, no output | The test reads STATUS and COUNT / len=0 | zero_len | Pass |
D4 | LEN 1 to 16 | Scoreboard / len=1, 2-4, 5-15, 16 | check; closure (16) | Pass merged; 16 not nightly |
D5 | LEN above 16, with every LEN bit toggled | Scoreboard / len>16 | closure (scratch) | Gap: fault 8 escaped |
E1 | Early TLAST ends the job: ERR not DONE, COUNT = words taken | ERR bit only / outcome=err, tlast=early | truncate, closure | Partial: DONE, COUNT unchecked |
E2 | A packet longer than LEN: exactly LEN words out, the rest never accepted | None / none | None | Not started |
E3 | An unmapped address answers PSLVERR and reads 0 | None / none (SLVERR items skipped) | None | Gap: code-coverage hole |
S1 | DONE rises on the cycle the last word is accepted (the output word may still wait for TREADY) | None / none | None | Gap: fault 6 escaped |
S2 | IRQ = IRQ_EN and (DONE or ERR), as a level | Tests wait on it / irq_en=1 | Most tests | Pass for IRQ_EN = 1 |
S3 | IRQ_EN = 0: no interrupt, the job still completes | Polled STATUS / irq_en=0 | closure (scratch) | Partial: not nightly |
S4 | W1C clears only bits written 1; ERR survives GO | Tests clear and read back / clear=done, clear=err | check, closure | Pass merged |
S5 | A clear in the cycle DONE sets: set wins | None / none | None | Not started |
R1 | Reset values and access policies | Built-in reset and bit-bash sequences | ral_reset, ral_bitbash | Pass |
R2 | LEN and KEY writes are ignored while BUSY | Mirror (LEN), scoreboard (KEY) / none | ral_busy_len, waiver | Gap: waived, DMA-142 |
R3 | GO is ignored while BUSY | None / none | None | Not started |
H1 | Output backpressure stalls the input with no loss | Scoreboard / none for stalls | backpressure | Pass, not covered |
H2 | Back-to-back input beats | Scoreboard / gap=0 | b2b driver (scratch) | Gap: driver limit |
X1 | Reset mid-job drops stale predictions; the next job is clean |
| reset | Pass |
Read the status column first: it changes every night, and it is where a review's real conversation happens. Note the two counts: the plan has nineteen rows, while the bench's coverage tally tracks seventeen items. Some rows close without a coverage item (R1 by the built-in register sequences, X1 by the reset test), and four rows are not started, which is exactly what the follow-up question is probing.
No check at all (D2, S1). The fault campaign forced TLAST to 0 and DONE to 1 and every test passed. Coverage sat near both behaviours, so they looked covered.
Out of the stimulus's reach (D5, S3, H2). LEN is constrained to 1-16, IRQ_EN is soft-constrained to 1, and the stream driver never sends back-to-back beats. The closure lesson closed all three with scratch tests that are not yet in the nightly.
Never written (E2, E3, R3, S5). PSLVERR on an unmapped address surfaced as a code-coverage hole, not a plan hole. A 100% plan beside code-coverage holes is usually an incomplete plan.
Exercised, not counted (E1), or checked wrongly (R2). dma_truncate_test runs on an env with no coverage component; the models take KEY and LEN writes while busy, which the DUT ignores (war story 9).
The coverage tally in dma_coverage prints, per test, which plan items were hit and which were not. This is dma_check_test, the main random test:
+UVM_TESTNAME=dma_check_test
UVM_INFO dma_tb.sv(1530) @ 1305000: uvm_test_top.env.cov [COV] plan items hit: 9 of 17
UVM_INFO dma_tb.sv(1531) @ 1305000: uvm_test_top.env.cov [COV] plan holes: len=0 len=16 len>16 irq_en=0 outcome=err clear=err gap=0 tlast=early
UVM_INFO dma_tb.sv(411) @ 1305000: uvm_test_top.env.sb [SB] 14 words matched, 0 mismatchedAnd these are rows 5 to 8 of the fault campaign's summary, which measures something coverage cannot: whether the checks would notice a broken design.
fault dma_rand_test dma_truncate_test
5 MISSED detected(E0,F1)
6 MISSED MISSED
7 MISSED MISSED
8 MISSED MISSED
faults detected by at least one test: 5 of 8The coverage holes map to rows by name. len>16 is D5, irq_en=0 is S3, gap=0 is H2. That mapping only works because the coverage items carry the plan's names.
The missed faults map to rows by behaviour: fault 6 is S1, fault 7 is D2, fault 8 is D5. None shows up as a coverage hole, which is why a plan needs a check column and a way to test the checks.
A good reviewer does not read a plan top to bottom admiring it. They look for what is missing, in roughly this order:
Trace the spec. Take the spec and count the sentences with no plan item. The DMA spec's "any other address completes with PSLVERR" had none.
Every row needs a check. Coverage without a check proves the scenario happened, not that the result was right. D2 and S1 fail this test.
Count the negative rows. Error paths, illegal accesses, writes at the wrong time, reset in the middle. Designers handle these last and tests reach them least; a plan that is mostly happy paths is a plan written from the stimulus side.
Look at how each row closes. Random, directed, formal or waived. A row that says random but sits outside the constraints (D5, S3) will never close by adding seeds.
Check status and reviewers. Every status comes from a named regression and build, and the designer, architect and software owner have each reviewed the plan. A plan reviewed only by its author is a draft.
A plan in a slide deck is stale the week it is written. A living testplan is a machine-readable file under version control, one record per item, that the nightly back-annotates with coverage, results and bug links. Vendors' planning tools do this with their coverage databases; for a small block a plain file works:
# dma_testplan.yaml (excerpt)
- id: D2
spec: "TLAST on the LEN-th output word"
check: dma_scoreboard.tlast_compare # to be added
cover: [tlast=on-time, tlast=early]
tests: [dma_check_test, dma_truncate_test]
closes_by: random + directed
status: gap # back-annotated nightly
ticket: DMA-151The bench's plan-item tally is the same idea in miniature. Three habits keep a living plan honest: IDs never change meaning once tests refer to them; a spec change produces a plan diff reviewed like code; and the nightly shows plan status beside the sign-off metrics, so nobody presents "17 of 17" without the check column.
The course's own bench is the war story. After the closure work, the merged regression hit 17 of 17 plan items, and the sign-off sheet said so. Then the fault campaign ran, and three permanent faults went straight through. The plan had a coverage item called tlast=on-time, sampled from the input stream, and a reader saw "TLAST" in the coverage report and assumed it was checked. Nothing compared the output TLAST against anything. The same was true of DONE: tests waited for the interrupt, which DONE raises, but no check said DONE must wait for the last word.
The fix is structural, not a new test. Every row gets a check column that names a specific comparison or assertion; D2 gets a TLAST compare in the scoreboard and S1 gets an assertion that DONE rises only on the cycle the last word is accepted. Then the fault campaign is rerun, and its result goes into the plan as a measure of the checks, the way coverage is a measure of the stimulus.
The check. Coverage shows the interrupt was enabled, not that it rose at the right time, stayed high as a level, or fell when DONE was cleared. Ask which component compares the interrupt against the spec equation (an interrupt monitor with a predictor, or an assertion on irq == irq_en && (done || err)), and ask for the IRQ_EN = 0 case, which this row never mentions.
assign pslverr = xfer && !mapped; never had pslverr high, but the plan is at 100%. What does that tell you?The plan is missing a row. Unmapped-address behaviour is in the spec, so this is not dead code to exclude: it is an untested feature. Add a plan item (E3) with a directed test that accesses an unmapped address and checks PSLVERR and the read data, and add an APB coverage item for SLVERR, which dma_coverage currently skips.
dma_truncate_test passes every night?Because the test checks only that ERR is set. The row promises more: DONE stays clear and COUNT equals the words taken. And the test runs on an env without the coverage component, so the error path is exercised but never counted. Passing tests do not make a row pass; the row's full check and coverage must both be present.
Next: a plan of twenty rows is also the best input you have for the question every manager asks next: how long will it take?