The software team wants to know, before tape-out, whether the operating system boots on the new chip. Booting takes a few billion clock cycles. The chip-level RTL simulation, the most accurate model the team has, runs at about a thousand cycles per second. The question is easy to state and, in simulation, takes about two months of wall-clock time for a single attempt.

Why do some verification questions need hardware platforms instead of simulation?

The obvious answer, and why it breaks

The obvious answer is to buy faster computers or more of them. Simulation of one design is mostly sequential: more machines run more tests in parallel, but each test is no faster, and a faster workstation buys perhaps a factor of two. The gap to software-scale workloads is six orders of magnitude. Closing it needs a different kind of platform: the design mapped onto hardware that runs it.

The idea: cycles divided by speed

Platform speeds differ by orders of magnitude (fact V12): whole-chip RTL simulation around a thousand cycles per second, emulation around a million, FPGA prototypes tens of millions, silicon about a billion.

Build it: a time-to-run calculator

Four workloads of increasing length (illustrative cycle counts) on four platforms at commonly quoted speeds.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

"""Why cycles matter: how long each platform takes to run the same software.

Workloads, in clock cycles of the chip (illustrative):
  driver init        2 million     a driver probing and configuring one block
  bare-metal test    50 million    a self-checking C test of a data flow
  OS boot            5 billion     booting an operating system to a shell
  benchmark          100 billion   a performance benchmark run
Platform speeds, in chip cycles per second, at the commonly quoted orders of magnitude
(fact V12): RTL simulation of a whole SoC 1 kHz, emulation 1 MHz, FPGA prototype 30 MHz,
silicon 1 GHz.
"""
work = {"driver init": 2e6, "bare-metal test": 5e7, "OS boot": 5e9, "benchmark": 1e11}
speed = {"RTL sim": 1e3, "emulation": 1e6, "FPGA proto": 3e7, "silicon": 1e9}

def human(s):
    for unit, size in (("years", 3.156e7), ("days", 86400), ("hours", 3600), ("min", 60), ("s", 1)):
        if s >= size:
            return f"{s / size:.1f} {unit}"
    return f"{s * 1000:.0f} ms"

print(f"{'workload':16s}" + "".join(f"{p:>13s}" for p in speed))
for w, cycles in work.items():
    print(f"{w:16s}" + "".join(f"{human(cycles / v):>13s}" for v in speed.values()))
boot_sim_days = work["OS boot"] / speed["RTL sim"] / 86400
boot_emu_hours = work["OS boot"] / speed["emulation"] / 3600
assert boot_sim_days > 30 and boot_emu_hours < 2
print(f"PASS: an OS boot takes {boot_sim_days:.0f} days in RTL simulation and {boot_emu_hours * 60:.0f} minutes on an emulator")

It prints:

workload              RTL sim    emulation   FPGA proto      silicon
driver init          33.3 min        2.0 s        67 ms         2 ms
bare-metal test    13.9 hours       50.0 s        1.7 s        50 ms
OS boot             57.9 days    1.4 hours      2.8 min        5.0 s
benchmark           3.2 years     1.2 days     55.6 min      1.7 min
PASS: an OS boot takes 58 days in RTL simulation and 83 minutes on an emulator

How to read it: a driver initialisation (2 million cycles) is feasible in simulation, about half an hour. A bare-metal data-flow test takes most of a day. An operating-system boot takes 58 days in simulation and 83 minutes on an emulator; a benchmark is out of reach for simulation altogether and takes a day even on an emulator. Each step to a faster platform costs accuracy, visibility or flexibility (lessons 10.3 and 10.4), so the right platform depends on the question.

Questions that need cycles

How real teams get their cycles

What breaks

A team (illustrative) planned to validate its boot flow in chip-level simulation "over the weekend". The boot needed several billion cycles; the simulation ran at a few hundred cycles per second. After a week it had reached the bootloader's second stage. A cycle-count estimate in the plan would have sent the boot test to the emulator from the start.

Key takeaways

• Wall-clock time is the workload's cycles divided by the platform's speed.

• Platform speeds differ by orders of magnitude: simulation, emulation, FPGA prototypes, silicon.

• Estimate questions in cycles and send long ones to faster platforms.

Related topics

• Part 10 overview: Emulation and Prototyping