A memory system is specified at 51.2 GB/s: a 64-bit interface at 6,400 million transfers per second. The team measures 35 GB/s on their workload and suspects a bug. There isn't one. 51.2 GB/s is the rate at which data moves while data is moving. The workload also spends time switching between reads and writes, opening new rows, and refreshing.
Why is sustained bandwidth below peak, and what decides how far below?
The obvious reading of a peak number is a promise: with enough traffic, that is what you get. But peak counts only the cycles in which data moves. Real traffic needs other cycles too: a shared bus must turn around between directions, a memory must open the right row before it can read it, and a dynamic memory must stop to refresh. Each of these is a fixed cost per event, so the shorter and more scattered the accesses, the more of the time goes to overhead.
Overhead | When it is paid | Made worse by |
|---|---|---|
Bus turnaround | Each switch between reading and writing on a shared data bus | Fine-grained mixing of reads and writes |
Row miss (page miss) | Each access to a row that is not already open | Scattered addresses, many streams |
Refresh | A fixed fraction of time, regardless of traffic | Nothing you control; larger, denser memories pay more |
Command and protocol gaps | Between accesses, per the device's timing rules | Short accesses |
Sustained bandwidth is beats moved divided by all the cycles taken, and every overhead above adds cycles that move nothing. The lever you control is traffic shape: how long each access is, and how often direction and row change.
The model is a DRAM-like bus with illustrative overheads: 4 idle cycles per change of direction, 10 cycles for a row miss on 30% of accesses, and 4% of time lost to refresh. Traffic is a random mix of reads and writes, two to one. It sweeps the number of beats per access.
"""Peak versus sustained bandwidth on a DRAM-like data bus (illustrative numbers).
The bus moves one beat per cycle at peak. Each access moves `burst` beats. Overheads:
- turnaround: 4 idle cycles whenever the direction changes (read <-> write)
- row miss: 10 extra cycles when an access opens a new row (here 30% of accesses)
- refresh: the bus is unavailable 4% of the time
Traffic: random reads and writes, 2:1. Sustained = beats moved / cycles taken.
"""
import random
TURN, MISS, P_MISS, REFRESH = 4, 10, 0.3, 0.04
def sustained(burst, accesses=20000):
random.seed(7)
cycles, beats, last = 0, 0, None
for _ in range(accesses):
op = "R" if random.random() < 2 / 3 else "W"
if last is not None and op != last:
cycles += TURN
if random.random() < P_MISS:
cycles += MISS
cycles += burst
beats += burst
last = op
return beats / (cycles / (1 - REFRESH))
bursts = [2, 4, 8, 16, 32, 64]
row = [sustained(b) for b in bursts]
print("beats per access: " + " ".join(f"{b:5d}" for b in bursts))
print("sustained/peak: " + " ".join(f"{x:5.2f}" for x in row))
assert all(a < b for a, b in zip(row, row[1:])) and row[-1] < 1 - REFRESH
print("PASS: fixed per-access overheads make short accesses expensive; even long ones stay below peak")It prints:
beats per access: 2 4 8 16 32 64
sustained/peak: 0.28 0.44 0.60 0.74 0.84 0.89
PASS: fixed per-access overheads make short accesses expensive; even long ones stay below peakTwo-beat accesses reach only 28% of peak: each pays the same turnaround and row-miss costs as a long access but moves almost nothing. At 64 beats the fixed costs are spread thin, yet refresh alone keeps the result below 96%.
Peak: 6400 million transfers/s x 8 bytes = 51.2 GB/s
Measured on the workload = 35 GB/s -> 68% of peak
In the model, 68% falls between 8- and 16-beat accesses: a mix of short, scattered accesses with frequent read/write switchingLonger accesses: whole cache lines or larger, and merged writes.
Batch by direction: memory controllers queue writes and drain them in batches, so the bus turns around less often.
Reorder for row hits: controllers schedule accesses to an open row first (within ordering rules, Part 7).
More banks and channels: interleave addresses so independent accesses overlap their overheads.
A design (illustrative) sized its memory interface from peak bandwidth: the accelerator needed 40 GB/s and the memory offered 51.2. In the product the accelerator's scattered 32-byte accesses sustained about half of peak, and it missed its frame rate. The fix combined longer accesses from a small on-chip buffer with an address layout that kept each stream in fewer rows; a wider memory would also have worked, at a cost.
8 / (8 + 6) ≈ 57%.
Bus turnaround: draining many writes together switches direction less often.
Peak bandwidth counts only cycles that move data; sustained bandwidth includes every overhead.
Turnaround, row misses and command gaps are paid per access, so short scattered accesses suffer most.
Size from sustained bandwidth at the real traffic shape, not from peak.