Ten bugs, ten tests, no orphans on either side: a bug with no test escapes, a test with no bug wastes the regression. The matrix below is the lab's coverage proof, and each cell names the mechanism, not just the stimulus, so a passing test that cannot catch its bug is visible as a weak cell.
Goal: a ten-row matrix with a catching test and an expected result per bug.
Inputs: the bug table in Planted Bugs and Grading, and the stage checkers, one per bug, each a function of the design data.
Steps: assign each bug its test; state when the test passes on clean data and how it fails on the bug; run the matrix checker, which injects every bug into a fresh copy of the design; confirm each test fails only its own bug.
Bug | Test | Passes on the clean design when | Fails on the bug with |
|---|---|---|---|
1 Overlap | Slot-select audit on every slot, plus the table overlap check | each probe address hits exactly its own slot | SBR probes hit DMA and SBR; KEY gets written |
2 Release order | Release times from the reset plan, and the first-fetch model | CPU leaves reset 8 clk_sys or more after rst_sys_n; the fetch survives | CPU 177 ns early; the fetch is lost |
3 Mis-wired IRQ | Force each source in turn, read RAW | each source lights its own bit only | RAW bit 6 instead of bit 1 |
4 Addressing | Read all five DMA registers through the wiring | CTRL, STATUS, LEN, KEY and COUNT come back | PSLVERR on three, STATUS for COUNT |
5 No sync | Crossing audit judged by width | every crossing has the mechanism its width needs | a bare irq_dma crossing is named |
6 Isolation | Clamp audit over all 114 signals | no signal unclamped | the two interrupt clamps are named |
7 Secure attr | Secure copy from SRAM1 through the SBR after boot | the SBR reaches SRAM1 | SLVERR on the first beat |
8 Lengths | A job with both lengths, then the driver check | DONE, no ERR, SBR not stuck, COUNT equals both LENs | ERR at word 32, or an SBR stuck BUSY |
9 Traps | Scenario with the driver: rekey while busy, clear after an error, start | no stale key, no stuck irq, IRQ_EN survives | all three symptoms from a naive driver |
10 Gate hang | Access to a gated, idle DMA | the access completes with PSLVERR | the access never completes |
MAP = [
("BOOTROM", 0x00000000, 0x4000),
("SRAM0", 0x20000000, 0x10000),
("SRAM1", 0x20010000, 0x10000),
("DMA", 0x40000000, 0x1000),
("SBR", 0x40001000, 0x1000),
("UART", 0x40002000, 0x1000),
("SPI", 0x40003000, 0x1000),
("GPIO", 0x40004000, 0x1000),
("INTC", 0x40005000, 0x1000),
("RSTPWR", 0x40006000, 0x1000),
]
SECURE_ONLY = {"SRAM1", "RSTPWR"}
SLOTS = ["DMA", "SBR", "UART", "SPI", "GPIO", "INTC", "RSTPWR"]
T_CPU, T_SYS, T_PER = 1e9 / 192e6, 1e9 / 96e6, 1e9 / 24e6
def end_of(base, size):
return base + size - 1
def overlaps(entries=MAP):
bad = []
xs = sorted(entries, key=lambda e: e[1])
for a, b in zip(xs, xs[1:]):
if b[1] < a[1] + a[2]:
bad.append((a[0], b[0]))
return bad
def misaligned(entries=MAP):
return [n for n, b, s in entries if b % s != 0]
def decode(addr, slots=SLOTS):
if addr >> 14 == 0:
return "BOOTROM"
if addr >> 16 == 0x2000:
return "SRAM0"
if addr >> 16 == 0x2001:
return "SRAM1"
if addr >> 16 == 0x4000:
slot = (addr >> 12) & 0xF
return slots[slot] if slot < 7 else "H3"
if addr < 0x20000000:
return "H1"
if addr < 0x40000000:
return "H2"
return "H4"
def response(addr, secure, slots=SLOTS):
t = decode(addr, slots)
if t in ("H1", "H2", "H4"):
return "DECERR"
if t == "H3":
return "SLVERR"
if t in SECURE_ONLY and not secure:
return "SLVERR"
return "OK"
def sustained(peak, beats):
return peak * beats / (beats + 1)
import hashlib
# ---- stage checkers: every one is a function of the design data d ----
ORACLE = {"irq_pwr": 0, "irq_dma": 1, "irq_sbr": 2, "irq_uart": 3, "irq_spi": 4, "irq_gpio": 5}
REG = {0x00: "CTRL", 0x04: "STATUS", 0x08: "LEN", 0x0C: "KEY", 0x10: "COUNT"}
GROUPS = {"apb_cdc": [("paddr", 12), ("pwdata", 32), ("psel", 1), ("penable", 1), ("pwrite", 1)],
"sram0": [("addr", 14), ("wdata", 32), ("be", 4), ("ce", 1), ("we", 1)],
"rom": [("addr", 12), ("ce", 1)], "irq": [("irq_dma", 1), ("irq_sbr", 1)]}
def signals():
return [g + "." + n + "[" + str(i) + "]" for g, items in GROUPS.items() for n, w in items for i in range(w)]
XING = [("X1", "clk_cpu", "clk_sys", "axi_req", 32, "fifo", 2), ("X2", "clk_sys", "clk_cpu", "axi_resp", 32, "fifo", 2),
("X3", "clk_sys", "clk_per", "apb_req", 32, "reqack", 2), ("X4", "clk_per", "clk_sys", "apb_resp", 32, "reqack", 2),
("X5", "clk_sys", "clk_per", "irq_dma", 1, "2ff", 2), ("X6", "clk_per", "clk_cpu", "irq_cpu", 1, "2ff", 2),
("X7", "clk_per", "clk_sys", "rst_sys_n", 1, "rstsync", 2), ("X8", "clk_per", "clk_cpu", "rst_cpu_n", 1, "rstsync", 2),
("X9", "pad", "clk_per", "gpio_in", 1, "2ff", 2), ("X10", "clk_per", "clk_sys", "pgood", 1, "2ff", 2)]
PLAN = {"rst_per_n": ("por", 1026 * T_PER), "rst_sys_n": ("gates", 18 * T_SYS), "rst_cpu_n": ("rst_sys_n", 8 * T_SYS + 2 * T_CPU)}
AREA = [0.150, 0.025, 0.015, 0.040, 0.012, 0.010, 0.006, 0.004, 0.100, 0.010,
0.100, 0.012, 0.008, 0.008, 0.006, 0.004, 0.015, 0.060]
POWER = [(77.10, 5.64), (11.25, 5.64), (0.16, 1.24)]
def design():
return {"map": list(MAP), "slots": list(SLOTS), "sel_mask": {}, "paddr": (0, 7), "xing": list(XING), "plan": dict(PLAN),
"irq": dict(ORACLE), "irq_sync": (2, 1, 2), "clamped": set(signals()), "wdog": 256, "sbr_ctrl": 0x4,
"boot_clears_ns": True, "ns_secure_only": False, "lens": (64, 64), "driver": "good", "cause_first": True,
"burst": 16, "link_width": 4, "fabric_mhz": 96, "banks": ("SRAM0", "SRAM1"), "clk": (24, 1, 32, 4, 8),
"area": list(AREA), "power": [tuple(p) for p in POWER]}
def selects(d):
sel = {n: (i << 12, d["sel_mask"].get(n, 0xF000)) for i, n in enumerate(d["slots"])}
return sel
def hits(sel, addr):
return [n for n, (v, m) in sel.items() if addr >> 16 == 0x4000 and (addr & 0xFFFF) & m == v]
def map_ok(d):
sel = selects(d)
slot_ok = all(hits(sel, 0x40000000 + (i << 12) + o) == [n] for i, n in enumerate(SLOTS) for o in (0, 0xC, 0xFFC))
return (overlaps(d["map"]) == [] and misaligned(d["map"]) == [] and slot_ok
and all(decode(b, d["slots"]) == n for n, b, s in d["map"]))
def dma_read(off, lo, hi):
if hi - lo != 7:
return "WIDTH"
return REG.get((off >> lo) & 0xFF, "PSLVERR")
def paddr_ok(d):
lo, hi = d["paddr"]
return [dma_read(o, lo, hi) for o in sorted(REG)] == ["CTRL", "STATUS", "LEN", "KEY", "COUNT"]
def audit(rows):
bad = []
for x, src, dst, sig, width, mech, flops in rows:
if sig.startswith("rst"):
good = mech == "rstsync" and flops >= 2
elif width == 1:
good = mech == "2ff" and flops >= 2
else:
good = mech in ("fifo", "reqack")
if not good:
bad.append(x)
return bad
def xing_ok(d):
return len(d["xing"]) == 10 and audit(d["xing"]) == []
def release_times(plan):
t = {"por": 0.0}
t["rst_per_n"] = plan["rst_per_n"][1]
t["gates"] = t["rst_per_n"] + 262 * T_PER
for net in ("rst_sys_n", "rst_cpu_n"):
after, wait = plan[net]
t[net] = t[after] + wait
return t
def fetch_lost(t):
return t["rst_cpu_n"] + T_CPU + 3 * T_SYS < t["rst_sys_n"]
def release_ok(d):
t = release_times(d["plan"])
return t["rst_per_n"] <= t["rst_sys_n"] and t["rst_cpu_n"] >= t["rst_sys_n"] + 8 * T_SYS and not fetch_lost(t)
def intc_raw(wiring, active):
return sum(1 << wiring[s] for s in active)
def conn_test(w):
return [s for s in ORACLE if intc_raw(w, [s]) != 1 << ORACLE[s]]
def latency(sync_per, ctrl, sync_cpu):
best = sync_per * T_PER + ctrl * T_PER + sync_cpu * T_CPU
return best, best + T_PER + T_CPU
def irq_ok(d):
best, worst = latency(*d["irq_sync"])
return conn_test(d["irq"]) == [] and abs(best - 135.4) < 0.1 and abs(worst - 182.3) < 0.1
def clamp_ok(d):
return [s for s in signals() if s not in d["clamped"]] == []
def apb_access(clk_en, wdog):
for cycle in range(1, 2000):
if clk_en and cycle >= 3:
return "OK"
if wdog and cycle >= wdog:
return "PSLVERR"
return "HANG"
def gate_ok(d):
return apb_access(False, d["wdog"]) == "PSLVERR" and apb_access(True, None) == "OK"
def sbr_ctrl_write(ctrl, wdata, writer_secure, ns_secure_only=False):
if ns_secure_only and not writer_secure:
wdata = (wdata & ~0x4) | (ctrl & 0x4)
return wdata & 0x6
def sbr_reaches_sram1(ctrl):
return response(0x20010000, (ctrl >> 2) & 1 == 0) == "OK"
def secure_ok(d):
ctrl = d["sbr_ctrl"]
if d["boot_clears_ns"]:
ctrl = sbr_ctrl_write(ctrl, 0x0, True, d["ns_secure_only"])
return sbr_reaches_sram1(ctrl)
def partition_closed(d):
return not sbr_reaches_sram1(sbr_ctrl_write(d["sbr_ctrl"], 0x0, False, d["ns_secure_only"]))
class Dma:
def __init__(self):
self.busy = self.done = self.err = self.irq_en = False
self.key = self.len = self.count = self.stalls = 0
def irq(self):
return self.irq_en and (self.done or self.err)
def write_key(self, v):
self.key = self.key if self.busy else v
def write_len(self, v):
self.len = self.len if self.busy else v & 0xFFFF
def ctrl_write(self, v):
self.irq_en = bool(v & 2)
if v & 1 and not self.busy:
if self.len == 0:
self.done = True
else:
self.busy, self.done, self.count = True, False, 0
def status_write(self, v):
self.done = self.done and not v & 2
self.err = self.err and not v & 4
def accept(self, last):
if not self.busy:
self.stalls += 1
return False
self.count += 1
if last and self.count < self.len:
self.err, self.busy = True, False
elif self.count == self.len:
self.done, self.busy = True, False
return True
def stream(dma, n):
return not all(dma.accept(i == n - 1) for i in range(n))
def job(n_dma, n_sbr, key=0x5A):
d = Dma()
d.write_key(key)
d.write_len(n_dma)
d.ctrl_write(0x3)
return d, stream(d, n_sbr)
def driver_ok(count, sbr_len, dma_len):
return count == sbr_len == dma_len and dma_len > 0
def len_ok(d):
n_dma, n_sbr = d["lens"]
dma, sbr_busy = job(n_dma, n_sbr)
return dma.done and not dma.err and not sbr_busy and driver_ok(dma.count, n_sbr, n_dma)
class Good:
def start(self, dma): dma.ctrl_write(0x3)
def rekey(self, dma, key, finish):
if dma.busy: finish(dma)
dma.write_key(key)
def clear(self, dma): dma.status_write(0x6)
class Naive:
def start(self, dma): dma.ctrl_write(0x1)
def rekey(self, dma, key, finish): dma.write_key(key)
def clear(self, dma): dma.status_write(0x2)
def contract_problems(drv):
found = []
a = Dma(); a.write_key(0xAA); a.write_len(4); a.ctrl_write(0x3); a.accept(False)
drv.rekey(a, 0x55, lambda x: [x.accept(i == 2) for i in range(3)])
if a.key != 0x55: found.append("stale_key")
b = Dma(); b.write_len(8); b.ctrl_write(0x3); b.accept(True)
drv.clear(b)
if b.irq(): found.append("irq_stuck")
c = Dma(); c.write_len(1); drv.start(c)
if not c.irq_en: found.append("irq_disabled")
return found
def contract_ok(d):
return contract_problems({"good": Good, "naive": Naive}[d["driver"]]()) == []
def link_table(d):
peak = d["link_width"] * d["fabric_mhz"]
s = sustained(peak, d["burst"])
return {"cpu_fill": 16 / (24 * T_CPU) * 1e3, "fabric": s, "sram_bank_shared": s / 2, "apb_fast": peak / 5,
"apb_slow": 4 / 208.33e-9 / 1e6}
def stream_rate(d):
t = link_table(d)
bank = t["sram_bank_shared"] if d["banks"][0] == d["banks"][1] else t["fabric"]
path = {"sbr_read": t["fabric"], "src_bank": bank, "dma_in": t["fabric"], "dma_out": t["fabric"],
"dst_bank": bank, "sbr_write": t["fabric"]}
low = min(path.values())
return low, sorted(n for n, v in path.items() if v - low < 0.05)
def links_ok(d):
t = link_table(d)
rate, tied = stream_rate(d)
want = {"cpu_fill": 128.0, "fabric": 361.4, "sram_bank_shared": 180.7, "apb_fast": 76.8, "apb_slow": 19.2}
return all(abs(t[k] - v) < 0.05 for k, v in want.items()) and abs(rate - 361.4) < 0.05 and abs(2 * rate - 722.8) < 0.1
def clocks_ok(d):
xtal, r, n, dc, ds = d["clk"]
vco = xtal * n / r
return 600 <= vco <= 1200 and (vco / dc, vco / ds, xtal) == (192.0, 96.0, 24) and abs(256 * T_PER / 1000 - 10.667) < 0.01
def boot_steps(cause, cause_first=True):
hw = [("osc", 1024 * T_PER), ("rst_per", 2 * T_PER), ("pll", 260 * T_PER), ("gates", 2 * T_PER),
("rst_sys", 18 * T_SYS), ("rst_cpu", 8 * T_SYS + 2 * T_CPU), ("fetch", 24 * T_CPU)]
rd, clr = ("read_cause", 54 * T_CPU), ("clear_sram0", 1024 * T_SYS)
if cause == "WAKE":
return hw + ([rd] if cause_first else [clr, rd])
rest = [("uart_init", 3 * 54 * T_CPU), ("banner", 160 / (24e6 / 208) * 1e9),
("dma_selftest", 5 * 125 + 7 * 125 + 72 * T_SYS + 26 * T_CPU + 6000), ("spi_load", 8196 * 8 / 6e6 * 1e9),
("checksum", 2048 * 18 * T_CPU)]
return hw + ([rd, clr] if cause_first else [clr, rd]) + rest
def boot_ok(d):
cold = sum(ns for _, ns in boot_steps("POR", d["cause_first"]))
wake = [n for n, _ in boot_steps("WAKE", d["cause_first"])]
return abs(cold / 1000 - 12580.917) < 0.01 and "clear_sram0" not in wake
# ---- the ten bugs: each injector breaks one fact of the design data ----
def inj1(d):
d["map"] = [("DMA", 0x40000000, 0x2000) if r[0] == "DMA" else r for r in d["map"]]
d["sel_mask"]["DMA"] = 0xE000
def inj2(d): d["plan"]["rst_cpu_n"] = ("gates", 2 * T_CPU)
def inj3(d): d["irq"]["irq_dma"] = 6
def inj4(d): d["paddr"] = (2, 9)
def inj5(d): d["xing"][4] = ("X5", "clk_sys", "clk_per", "irq_dma", 1, "direct", 0)
def inj6(d): d["clamped"] -= {s for s in signals() if s.startswith("irq.")}
def inj7(d): d["boot_clears_ns"] = False
def inj8(d): d["lens"] = (64, 32)
def inj9(d): d["driver"] = "naive"
def inj10(d): d["wdog"] = None
BUGS = {1: ("map", map_ok, inj1), 2: ("release", release_ok, inj2), 3: ("irqs", irq_ok, inj3), 4: ("paddr", paddr_ok, inj4),
5: ("crossings", xing_ok, inj5), 6: ("clamps", clamp_ok, inj6), 7: ("secure", secure_ok, inj7),
8: ("lengths", len_ok, inj8), 9: ("contract", contract_ok, inj9), 10: ("gate", gate_ok, inj10)}
# ---- this lesson starts here ----
def breaks(fn, inj):
d = design()
inj(d)
return not fn(d)
OWNER = {1: 1, 4: 1, 2: 2, 5: 2, 6: 2, 10: 2, 3: 3, 7: 3, 8: 3, 9: 3}
assert sorted(BUGS) == list(range(1, 11)) and sorted(OWNER) == list(range(1, 11))
for bug, (name, fn, inj) in BUGS.items():
assert fn(design()), name # the test passes on the clean design
assert breaks(fn, inj), name # and fails once its bug is injected
d = design()
inj(d)
failed = {b for b, (n, f, i) in BUGS.items() if not f(d)}
assert failed == {bug}, (bug, failed) # no other test notices: each result identifies its bug
assert not breaks(lambda d: True, BUGS[3][2]) # a test that cannot fail is refused by the same rule
per_stage = [sum(1 for v in OWNER.values() if v == s) for s in (1, 2, 3)]
assert per_stage == [2, 4, 4] and all(per_stage)
print("PASS: matrix covers 10 of 10 bugs; each test passes clean and fails only its own bug; stages own 2, 4 and 4")One PASS line. The loop is the matrix: it builds a fresh design, runs each test on it, injects the bug into another fresh design and requires the test to fail. The failed-set assert shows each bug is identified by exactly one test. The lambda that always returns True is the weak test the pitfall describes: the same rule would refuse it.
A clean-design failure means the design data and the test disagree: fix the data. A breaks failure means the injector does not touch what the test reads, or the test compares against the bug's own data. A failed-set mismatch means two tests share an observation and one test cannot name its bug. An ownership failure means a stage owns nothing, which breaks the lab rule that every stage earns its keep.
Hint 1: sort the bugs by stage first; the owners follow the bug table in Planted Bugs and Grading. Hint 2: structural bugs need structural tests: audits over the data, not waveforms. Hint 3: contract bugs need sequences against a model, not single accesses. Answer: the ten pairs above, with stages owning 2, 4 and 4, and each test failing on its own bug alone.
Without the watchdog the access hangs, which is the bug; with it the access completes with PSLVERR, which is the fix. A test that has only ever run with the fix present has never been shown to fail.