AMAT combines hit time, miss rate and miss penalty. A lower miss rate can still lose IPC when it buys longer hit latency or a heavier penalty.
Memory-level parallelism further discounts raw miss counts. Overlapping misses share one penalty, so exposed latency decides throughput, not total misses.
Performance follows miss penalty, overlap and any hit-latency shift, not the rate. A larger but slower cache can cut the rate while raising AMAT and hurting IPC.
Look for surges in invalidations, replays and retries plus protocol-state anomalies. Plain contention shows queue pressure without that protocol signature.
AMAT from stated parameters
formula: AMAT = hit_time + miss_rate x miss_penalty
example: hit_time = 2 cycles, miss_rate = 4 percent, miss_penalty = 180 cycles
result: 2 + 0.04 x 180 = 2 + 7.2 = 9.2 cycles per accessThe model below checks AMAT arithmetic against hand calculation. It is an arithmetic check, not a hardware model.
module amat_model (
input logic [15:0] hit_cycles,
input logic [15:0] miss_per100,
input logic [15:0] miss_penalty,
output logic [31:0] amat_x100
);
always_comb amat_x100 = hit_cycles * 100 + miss_per100 * miss_penalty;
endmodule
module tb_amat;
logic [15:0] hit_cycles;
logic [15:0] miss_per100;
logic [15:0] miss_penalty;
logic [31:0] amat_x100;
amat_model dut (.*);
int bad = 0;
initial begin
hit_cycles = 2; miss_per100 = 4; miss_penalty = 180; #1;
if (amat_x100 != 920) bad++;
hit_cycles = 3; miss_per100 = 2; miss_penalty = 120; #1;
if (amat_x100 != 540) bad++;
if (bad == 0) $display("PASS: amat gives 9.20 and 5.40 cycles for the two hand cases");
else $display("FAIL: %0d checks wrong", bad);
$finish;
end
endmoduleAssociativity trades conflict misses against hit latency, power and tail latency. Stop adding ways when marginal MPKI gain turns negative against those costs.
Prefetch pollution and demand misses need separate buckets. Pollution calls for admission and timeliness tuning, while demand misses call for structure or mapping changes.
Small MPKI movement together with measurable hit-latency, power or tail-latency regression on target workloads. The structure then costs more than it saves.
Set-level pressure heatmaps plus short line-lifetime spikes aligned with MPKI bursts. Aggregate misses alone cannot localize the hot sets.
On streaming or irregular access, useless prefetches evict useful lines and waste bandwidth. Measure MPKI and bandwidth with prefetch on and off before judging.
It shows whether evicted lines had near-term reuse, which guides insertion and victim policy. MPKI alone cannot distinguish capacity from policy failures.
Observation | Likely cause | Experiment |
|---|---|---|
MPKI burst on few sets | Conflict or thrashing | Raise associativity or re-map and re-measure |
MPKI up with prefetch on | Pollution | Gate admission and compare bandwidth |
Flat misses, higher latency | Hit-time or penalty shift | Recompute AMAT per level |
Add a level when MPKI times penalty times frequency justifies area, power and the extra hit latency. More levels are never free.
Inclusion simplifies coherence through back-invalidation but evicts outer lines needlessly. The choice is workload dependent, not doctrinal.
When the stall product, MPKI times miss penalty, converted to cycles per instruction exceeds the added hit latency plus area and power budget. Prototype with the penalty math first.
Inclusive hierarchies simplify coherence but cause eviction traffic in outer levels. Exclusive or non-inclusive designs save capacity at the cost of harder coherence bookkeeping.
Each miss triggers a page walk of tens to hundreds of cycles and stalls the pipeline. Sparse pointer-chasing workloads can spend more time walking than accessing data.
Stall cost of an L2 miss stream
formula: stall_CPI = MPKI x miss_penalty / 1000
example: MPKI = 8, miss_penalty = 120 cycles, fully exposed without overlap
result: 8 x 120 / 1000 = 960 / 1000 = 0.96 cycles per instructionMPKI by level, bandwidth utilization per hop, and the stall breakdown into frontend, backend and memory. Those three separate structure, throughput and overlap effects.