Each row exists because of a risk in the plan, and each is sampled from transactions or derived state, as in Transaction and Temporal Coverage.
Row | What is counted | Risk it answers | Goal |
|---|---|---|---|
CV1 | Reads outstanding when a read is accepted: 0, 1-3, 4-7, 8+ | Empty and full table; 8+ means this read took the last free entry (any index) | 5, 10, 10, 3 |
CV1b | Read allocated table entry 7 (white-box: the DUT's free_idx) | The highest entry index is used; depth alone does not show which entry was taken | 3 |
CV2 | Read accepted with 2 or more reads outstanding on its own ID | Within-ID ordering only matters when a queue has depth | 10 |
CV3 | Response that overtook an older read on another ID | Proves reordering happened, so ordering checks had work | 10 |
CV4 | Partial writes per port and byte lane (8 bins) | Strobe handling per port | 5 each |
CV5 | Read of a word written 3 or fewer cycles earlier | Ordering point between writes and reads | 10 |
CV6 | Response stalled 3 or more cycles | Payload hold under back-pressure | 5 |
CV7 | Contention won by port 0, by port 1 | Arbitration both ways | 5 each |
CV8 | Reset with 4 or more reads outstanding | Reset that actually drops work | 1 |
CV9 | Read on an ID that had reads outstanding at the last reset | State that survives reset | 1 |
CV3 and CV2 are activity gates as much as coverage. A clean scoreboard on traffic that never reordered, or never stacked two reads on one ID, proves nothing about R1.
On a commercial simulator the read and write rows are covergroups sampled from the monitor. Verilator 5.048 ignores covergroups, so the bench uses a procedural mirror (cap_cov) with the same bins and checks its rows against their goals, the approach of the Coverage Modelling topic.
// not-run: covergroups are parsed but ignored by Verilator 5.048 (COVERIGN warnings, get_coverage() returns 0) and rejected by Icarus 13; needs a commercial simulator
// The capstone coverage model as covergroups. The bench runs a procedural mirror of it.
class cap_cg;
covergroup cg_read with function sample(int depth, int depth_id, bit raw_close, bit after_reset);
option.per_instance = 1;
cp_depth: coverpoint depth { // CV1: table the read found
bins empty = {0};
bins low = {[1:3]};
bins high = {[4:7]};
bins full = {[8:$]}; // this read took the last free entry
}
cp_same_id: coverpoint depth_id { // CV2
bins deep = {[2:$]};
}
cp_raw: coverpoint raw_close { bins hit = {1}; } // CV5
cp_reuse: coverpoint after_reset { bins hit = {1}; } // CV9
x_depth_same: cross cp_depth, cp_same_id; // ordering under load
endgroup
covergroup cg_write with function sample(bit port, bit [3:0] strb);
cp_port: coverpoint port;
cp_lane: coverpoint strb iff (strb != 4'hF) { // CV4: partial writes only
wildcard bins lane0 = {4'b???1}; // overlapping bins: a write
wildcard bins lane1 = {4'b??1?}; // counts once in every lane
wildcard bins lane2 = {4'b?1??}; // it touches
wildcard bins lane3 = {4'b1???};
}
x_port_lane: cross cp_port, cp_lane;
endgroup
// White-box: which table entry a read was allocated (DUT's free_idx at a read
// acceptance). Depth says how busy the table was, not which entry was used.
covergroup cg_alloc with function sample(bit [2:0] slot);
cp_slot: coverpoint slot { // CV1b
bins last = {7};
bins other = {[0:6]};
}
endgroup
function new();
cg_read = new();
cg_write = new();
cg_alloc = new();
endfunction
endclassThe lane bins overlap on purpose. A write with strobe 0101 counts once in lane 0 and once in lane 2, which is what the mirror's loop over set strobe bits does. Bins that partitioned the strobe values instead would put that write in an "other" bin and leave lane 2 looking untested.
Rows that describe a shape in time read better as cover property, labelled with their row IDs. CV6 and CV7 could be written either way. The last two read naturally as sequences, although each also has a counter form (the mirror counts overtakes as CV3). cv11_overtake is only an approximation: the other-ID response may belong to a read older than the ID 1 read, and the final ID 1 response may serve an older ID 1 read. An exact overtake needs per-read tickets, which the CV3 mirror keeps.
// snippet: temporal plan rows as cover properties, labelled with their row IDs
cv6_stall3: cover property (@(posedge clk) disable iff (!rst_n)
(rs.valid && !rs.ready) [*3] ##1 (rs.valid && rs.ready)); // CV6: held 3 cycles, then taken
cv7_p1_wins: cover property (@(posedge clk) disable iff (!rst_n)
rq0.valid && rq1.valid && rq1.ready); // CV7: contention, port 1 wins
cv10_b2b_same_id: cover property (@(posedge clk) disable iff (!rst_n)
rsp_hs ##1 (rsp_hs && $stable(rsp_pid))); // two responses back to back on one ID
// Approximation: the other-ID response may belong to an older read, and the last
// response may serve an older ID 1 read. An exact overtake needs tickets (CV3 mirror).
cv11_overtake: cover property (@(posedge clk) disable iff (!rst_n)
(rd_acc && acc_pid == 3'd1) ##[1:8] (rsp_hs && rsp_pid != 3'd1) ##[1:8] (rsp_hs && rsp_pid == 3'd1));Verilator evaluates cover property only when built with --coverage-user, and the course runner does not pass it; Assertion-Backed Coverage shows the sentinel-cover check that tells "not reached" from "not evaluated". The capstone keeps these rows in the mirror so the runnable bench enforces them.
The full bench (300 items per port per phase, then a directed empty-table step) meets every goal in one run on every seed we tried: 1 to 30 (11 is the default) and 99. To see closure work, the bench was run with +items=20 +empty_reads=0 (the directed step off, as the bench was first written) and seeds 1 to 10, 104 to 116 cycles each. No single run met every goal; each left 4 to 8 rows short. The table is the measured mirror output; the bench below merges it.
// Closure over 10 short capstone runs (+items=20 +empty_reads=0, +seed=1..10). The hit
// counts are the coverage-mirror rows those runs printed; this bench merges them.
module tb_capstone_closure;
localparam int SEEDS = 10, ROWS = 21;
string row_name [ROWS] = '{
"CV1 depth 0", "CV1 depth 1-3", "CV1 depth 4-7", "CV1 depth 8+ (full)", "CV1b entry 7",
"CV2 same-ID depth >= 2", "CV3 cross-ID overtake",
"CV4 p0 lane 0", "CV4 p0 lane 1", "CV4 p0 lane 2", "CV4 p0 lane 3",
"CV4 p1 lane 0", "CV4 p1 lane 1", "CV4 p1 lane 2", "CV4 p1 lane 3",
"CV5 read-after-write", "CV6 stall >= 3", "CV7 p0 wins", "CV7 p1 wins",
"CV8 reset busy", "CV9 reuse after reset"};
int goal [ROWS] = '{5, 10, 10, 3, 3, 10, 10, 5, 5, 5, 5, 5, 5, 5, 5, 10, 5, 5, 5, 1, 1};
int hits [SEEDS][ROWS] = '{
'{2, 25, 19, 1, 1, 6, 11, 12, 11, 8, 6, 11, 12, 7, 7, 5, 2, 42, 44, 1, 4},
'{2, 29, 18, 0, 0, 6, 9, 10, 10, 4, 7, 14, 10, 14, 9, 6, 0, 44, 46, 1, 3},
'{2, 14, 40, 3, 3, 7, 17, 10, 9, 6, 5, 9, 7, 4, 4, 11, 4, 44, 46, 1, 3},
'{2, 22, 30, 1, 1, 9, 13, 10, 11, 12, 6, 14, 7, 5, 6, 10, 3, 44, 46, 1, 3},
'{2, 19, 33, 4, 2, 5, 11, 14, 7, 5, 4, 9, 7, 4, 3, 3, 1, 43, 44, 1, 4},
'{2, 22, 23, 2, 2, 9, 15, 14, 14, 9, 7, 8, 12, 6, 8, 7, 3, 44, 46, 1, 3},
'{3, 17, 34, 0, 2, 8, 16, 13, 7, 7, 5, 10, 12, 9, 10, 3, 2, 44, 46, 1, 3},
'{2, 19, 29, 3, 2,14, 13, 12, 7, 6, 4, 8, 11, 4, 6, 6, 3, 42, 45, 1, 2},
'{2, 21, 26, 1, 1, 4, 10, 12, 8, 12, 8, 13, 11, 8, 7, 6, 6, 43, 45, 1, 4},
'{3, 22, 20, 4, 3, 4, 17, 8, 8, 2, 9, 9, 10, 9, 11, 12, 3, 42, 44, 1, 4}};
// Rows below goal for one set of counts.
function automatic int holes(int h [ROWS]);
int n = 0;
for (int r = 0; r < ROWS; r++) if (h[r] < goal[r]) n++;
return n;
endfunction
// Total shortfall: how many hits are still missing across all rows.
function automatic int deficit(int h [ROWS]);
int d = 0;
for (int r = 0; r < ROWS; r++) if (h[r] < goal[r]) d += goal[r] - h[r];
return d;
endfunction
initial begin
int per_seed [SEEDS], curve [SEEDS], merged [ROWS], ranked [ROWS], order [$];
int errors = 0;
bit used [SEEDS];
merged = '{default: 0}; ranked = '{default: 0};
// 1. Each run on its own.
for (int s = 0; s < SEEDS; s++) per_seed[s] = holes(hits[s]);
$display("holes per run: %p", per_seed);
// 2. Rows that no single run closes: a stimulus question, not a seed question.
for (int r = 0; r < ROWS; r++) begin
bit always_hole = 1;
for (int s = 0; s < SEEDS; s++) if (hits[s][r] >= goal[r]) always_hole = 0;
if (always_hole) $display("below goal in every run: %s", row_name[r]);
if (always_hole != (r == 0)) errors++;
end
// 3. Merge counts run by run (what a coverage merge does with bin hits).
for (int s = 0; s < SEEDS; s++) begin
for (int r = 0; r < ROWS; r++) merged[r] += hits[s][r];
curve[s] = holes(merged);
end
$display("rows below goal after merging 1..10 runs: %p", curve);
// 4. Greedy ranking: take the run that removes most of the shortfall.
while (deficit(ranked) > 0) begin
int best = -1, best_d = 0;
for (int s = 0; s < SEEDS; s++) begin
int t [ROWS], d;
if (used[s]) continue;
for (int r = 0; r < ROWS; r++) t[r] = ranked[r] + hits[s][r];
d = deficit(t);
if (best < 0 || d < best_d) begin best = s; best_d = d; end
end
used[best] = 1;
for (int r = 0; r < ROWS; r++) ranked[r] += hits[best][r];
order.push_back(best + 1);
$display("rank %0d: seed %0d, shortfall left %0d", order.size(), best + 1, best_d);
end
// Results recomputed independently (Python) from the same table.
if (per_seed != '{6, 8, 5, 5, 8, 6, 6, 6, 5, 4}) errors++;
if (curve[0] != 6 || curve[1] != 4 || curve[2] != 0 || curve[9] != 0) errors++;
if (order.size() != 2 || order[0] != 3 || order[1] != 7) errors++;
if (errors == 0)
$display("PASS: no single run closes; 3 merged runs close all %0d rows; seeds 3 and 7 alone close them", ROWS);
else $fatal(1, "FAIL: %0d closure results differ", errors);
$finish;
end
endmoduleThe bench passes and prints the closure curve: 6 rows short after one run, 4 after two, none after three. Ranking finds that seeds 3 and 7 together close every row. One row, CV1 depth 0, is below goal in all ten runs, with 2 or 3 hits each.
Mutant run. Merging pass/fail flags instead of hit counts, so a row counts as closed only if some single run met its goal, gives a curve that ends at 1 row short after ten runs, and the bench fails. CV1 depth 0 never closes under that rule.
Which merge is right depends on what the row means. "The table was found empty at least five times across the regression" is a count, and counts add. "Every test drains the table at least once" is a per-test property, and only flags express it. Decide per row and write it in the plan; the two answers differ here.
CV1 depth 0 is the more useful finding. It is short in every run because the stimulus almost never lets the table empty: drivers send back to back, and gaps are short. More seeds will not fix a structural gap. The full-length run reached the goal on most seeds only because it is longer: without the directed step, 6 of 31 seeds (seeds 1 to 30 and 99; failing: 4, 7, 8, 10, 26 and 99) ended with 3 or 4 hits against a goal of 5, so the gate would fail about one correct run in five. The fix is the directed scenario now in the bench as phase D: wait for the table to empty, send one read, five times. With it, all 31 seeds pass, with 8 to 18 hits on the row. The arithmetic for when more seeds will help is in Closure at Scale.
Holes can also be the only sign of a bug. Two experiments with the full bench:
Experiment | Checkers | Coverage |
|---|---|---|
| None fired; the bug is never exercised. Only the bench's own check that a reset happened fails | CV8 and CV9 at 0: holes |
| None fired; port 1 is still served when port 0 is idle | CV7 port 1 wins at 0: hole |
In both cases a reviewer who waived the hole as "rare" would have signed off a broken design. A hole on a row that the plan ranked high needs a reason, not a waiver.
They prove the conditions under which the ordering checks mean something actually occurred: two reads queued on one ID, and a response overtaking an older one. Without them, a clean scoreboard could come from traffic that never tested ordering.
The one the plan row specifies. If the row only needs the empty-table case to be exercised a few times in the regression, the count merge is correct and the row is closed. If it means every test must drain the table, the row is open, and the fix is stimulus, not seeds.