Average memory access time adds hit time to the miss product. AMAT equals hit time plus miss rate times miss penalty, all in the same cycle unit.
The worked example below uses illustrative inputs. A better hit rate still loses when the hit itself slows down.
single level, illustrative: hit 1 cycle at 95%, next level 12 cycles, DRAM 180 cycles
miss rate = 1 - 0.95 = 0.05
miss penalty = 12 + 0.2 x 180 = 12 + 36 = 48 cycles
AMAT = 1 + 0.05 x 48 = 1 + 2.4 = 3.4 cycles
wider variant: hit 2 cycles at 96.5%
AMAT = 2 + 0.035 x 48 = 2 + 1.68 = 3.68 cycles
verdict: slower, despite the better rateEach level nests inside the miss penalty of the one above. The recursion stops at DRAM time.
AMAT equals L1 hit time plus L1 miss rate times the rest, where the rest is L2 hit time plus L2 local miss rate times the rest below it.
illustrative: L1 2 cycles, local miss 6%; L2 12 cycles, local miss 25%;
LLC 36 cycles, local miss 40%; DRAM 200 cycles
inner = 36 + 0.40 x 200 = 36 + 80 = 116 cycles
middle = 12 + 0.25 x 116 = 12 + 29 = 41 cycles
AMAT = 2 + 0.06 x 41 = 2 + 2.46 = 4.46 cyclesLevel | Hit time | Local miss rate | Role in the sum |
|---|---|---|---|
L1 | 2 cycles | 6% | Paid on every access |
L2 | 12 cycles | 25% of L1 misses | Paid on L1 misses only |
LLC | 36 cycles | 40% of L2 misses | Paid on L2 misses only |
DRAM | 200 cycles | Remainder | Paid on LLC misses only |
Miss rate divides by accesses, while MPKI divides by a thousand instructions. Converting needs the access count per kilo-instruction, which IPC alone cannot supply.
Overlap then discounts the penalty. Misses served in parallel cost less than AMAT assumes, which is why non-blocking caches carry miss-status registers and banks.
L1D miss rate 5%, 60 L1D accesses per 1000 instructions
MPKI = 0.05 x 60 = 3.0 misses per kilo-instruction
no-overlap stall = 3.0 / 1000 x 48 cycles = 0.144 cycles per instruction
with 2x overlap the effective penalty halves before it reaches CPITechnique | What it overlaps or serves | Cost |
|---|---|---|
Non-blocking cache with MSHRs | Several misses in flight at once | MSHR entries and replay logic |
Banking and extra ports | Parallel hits to different banks | Area and arbitration delay |
Critical-word first with early restart | The needed word before the full line | Wider fill datapath |
Hierarchical AMAT. It multiplies the 3-cycle hit cost by every L1 miss and the rate gain by the deeper penalty, in one number.
Their access counts per kilo-instruction differ. MPKI folds in access intensity, so equal rates can mean unequal instruction cost.