DCRouter's input space was packets on four ports, and still it took six lessons of stimulus, checking and coverage to catch 24 bugs. A processor's input space is every program that could ever run on it, in every privilege mode, with interrupts arriving at any instruction. Yet processor teams have one large advantage DCRouter's team never had: an instruction-set architecture that defines, precisely and publicly, the result of every instruction.
How are processors verified, given that their input space is every program?
The obvious approach is to run real programs (benchmarks, an operating system boot) and check that they produce correct output. Real programs are a good final test, but they use a narrow slice of the instruction set, rarely hit hazards in unusual orders, and check only their final output, so a wrong intermediate result that does not change the output, or a bug in a rarely used instruction, is missed. And when the output is wrong after a billion instructions, finding the first wrong one is the hard part.
Element | Role |
|---|---|
Instruction-set simulator (ISS) | The reference model: executes each instruction as the ISA defines it |
Lock-step comparison | After every retired instruction, compare PC, register and memory writes and traps; a mismatch points at the first wrong instruction |
Random instruction generators | Programs constrained to be legal, biased towards hazards, branches, exceptions and unusual operands (riscv-dv is an open example, fact V16) |
Compliance suites | Directed tests that every instruction behaves as the ISA specifies (riscv-arch-test, fact V16) |
Formal on the pipeline | Forwarding, stalls and hazards proved, often by comparing against a non-pipelined model |
Real software | Boots and benchmarks on emulation and prototypes (Part 10) |
This is DCRouter's reference model and checked random hunt (lessons 5.5 and 5.10) scaled up, with an oracle someone else has already written precisely. Lock-step comparison is the processor equivalent of comparing every packet rather than the final counts: the earlier the check, the closer the failure report is to the bug.
Microarchitecture: branch predictors, caches and speculation are invisible in the ISS; bugs that only affect performance need performance checks (lesson 8.7), and leaks need security properties (lesson 15.2).
Asynchronous events: interrupts and debug requests arrive at arbitrary points; the ISS must be told when the RTL took them, then check the effects.
Implementation-defined behaviour: where the ISA allows choices, the ISS must be configured to match the design's choices, or the comparison reports false mismatches.
Memory ordering with several cores: no single sequential reference exists; ordering is checked against the memory model's rules instead.
The ISS itself: it can be wrong; compliance suites and a second model help.
Practice | Purpose |
|---|---|
Lock-step ISS comparison in every random test | Every instruction checked, failures localised |
Generators biased by coverage of hazards, exceptions and privilege transitions | Rare pipeline interactions exercised |
Compliance suites as a release gate | Every instruction meets the ISA |
OS boots and benchmarks on emulation | Long, realistic programs and software interaction |
A team (illustrative) compared only the final register file at the end of each random program. A forwarding bug occasionally gave a wrong value to an instruction whose result was later overwritten, so the final state was correct and the bug survived months of regressions. Switching to lock-step comparison at every retired instruction exposed it in the first night's run.
A wrong intermediate value can be overwritten before the end, and when the final state is wrong, finding the first wrong instruction is hard; lock-step reports the first divergence where it happens.
The ISS models architectural results, not timing; a performance-only bug produces correct results, so it needs performance checks.
A processor's ISA is a precise, ready-made oracle.
Random programs, an ISS in lock-step and compliance suites form the core strategy.
Plan separately for microarchitecture, asynchronous events and multi-core ordering.